Image processing apparatus, control method, and storage medium

ABSTRACT

A control method for an image processing apparatus that generates a plurality of pieces of album data corresponding to a plurality of albums includes automatically selecting at least one piece of image data out of a plurality of pieces of image data as common image data corresponding to an image to be placed in a common page, which is a page common to all of the plurality of albums, based on an evaluation that is based on a first factor, and automatically selecting at least one piece of image data out of the plurality of pieces of image data as individual image data corresponding to an image to be placed in an individual page, which is a page not common to all of the plurality of albums, based on an evaluation that is based on a second factor.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.16/787,401, filed Feb. 11, 2020, which claims the benefit of JapanesePatent Application No. 2019-036830, filed Feb. 28, 2019, both of whichare hereby incorporated by reference herein in their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

Aspects of the present invention generally relate to an image processingapparatus, a control method, and a storage medium.

Description of the Related Art

In recent years, there have been known methods for generating album dataused to output an album in which images are arranged. Japanese PatentApplication Laid-Open No. 2016-66328 discusses a method in which aplurality of users edit album data in collaboration with each other.

However, Japanese Patent Application Laid-Open No. 2016-66328 does nottake into consideration processing which is performed to generate aplurality of pieces of album data including a common page, which is apage common to all of the plurality of pieces of album data, and anindividual page, which is a page not common to all of the pieces ofalbum data.

SUMMARY OF THE INVENTION

Aspects of the present disclosure are generally directed toappropriately generating a plurality of pieces of album data including acommon page, which is a page common to all of the plurality of pieces ofalbum data, and an individual page, which is a page not common to all ofthe pieces of album data.

According to an aspect of the present disclosure, a control method foran image processing apparatus that generates a plurality of pieces ofalbum data corresponding to a plurality of albums includes acquiring aplurality of pieces of image data, evaluating each of the plurality ofpieces of image data based on a first factor, evaluating each of theplurality of pieces of image data based on a second factor differentfrom the first factor, automatically selecting at least one piece ofimage data out of the plurality of pieces of image data as common imagedata corresponding to an image to be placed in a common page, which is apage common to all of the plurality of albums, based on an evaluationthat is based on the first factor, generating common page datacorresponding to the common page based on the at least one piece ofimage data selected as the common image data, automatically selecting atleast one piece of image data out of the plurality of pieces of imagedata as individual image data corresponding to an image to be placed inan individual page, which is a page not common to all of the pluralityof albums, based on an evaluation that is based on the second factor,generating a plurality of pieces of individual page data correspondingto the individual page based on the at least one piece of image dataselected as the individual image data, and generating the plurality ofpieces of album data based on the common page data and the plurality ofpieces of individual page data, wherein the individual page datacorresponding to first album data out of the plurality of pieces ofalbum data and the individual page data corresponding to second albumdata out of the plurality of pieces of album data are different fromeach other, and wherein the common page data corresponding to the firstalbum data out of the plurality of pieces of album data and the commonpage data corresponding to the second album data out of the plurality ofpieces of album data are identical to each other.

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a hardware configuration of animage processing apparatus.

FIG. 2 is a diagram illustrating examples of spreads included in albumswhich are created by an album creation application.

FIG. 3 is a diagram illustrating a setting screen which is displayed bythe album creation application.

FIGS. 4A, 4B, and 4C are flowcharts illustrating automatic layoutprocessing which is performed by the album creation application.

FIGS. 5A, 5B, and 5C are diagrams used to explain segmentation of animage data group.

FIG. 6 is a diagram used to explain classification of scenes.

FIGS. 7A, 7B, 7C, 7D, 7E, 7F, 7G, 7H, and 7I are diagrams used toexplain selection of image data.

FIG. 8 is a flowchart illustrating processing for determining a templatewhich is used to generate a layout image.

FIG. 9 is a diagram used to explain layouts of image data.

FIG. 10 is a flowchart illustrating spread joining processing which isperformed by the album creation application.

FIGS. 11A, 11B, and 11C are flowcharts illustrating automatic layoutprocessing which is performed by the album creation application.

FIGS. 12A, 12B, and 12C are flowcharts illustrating automatic layoutprocessing which is performed by the album creation application.

FIGS. 13A, 13B, 13C, and 13D are flowcharts illustrating automaticlayout processing which is performed by the album creation application.

FIGS. 14A and 14B are diagrams used to explain a cover setting area.

FIGS. 15A, 15B, 15C, and 15D are diagrams illustrating examples oftemplates for cover.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the disclosurewill be described in detail below with reference to the drawings.

In the following exemplary embodiments, procedures for activating anapplication program for creation of an album (hereinafter referred to asan “album creation application”) and performing automatic layoutprocessing to generate a layout image are described. The term “image” asused in the following description includes, unless otherwise stated, astill image, a moving image, and a frame image included in a movingimage, and further includes a still image, a moving image, and a frameimage included in a moving image which are present on a socialnetworking service (SNS) server.

FIG. 1 is a block diagram illustrating a hardware configuration of animage processing apparatus according to a first exemplary embodiment ofthe present disclosure. Examples of the image processing apparatusinclude a personal computer (PC), a smartphone, a tablet terminal, acamera, and a printer, and, in the first exemplary embodiment, the imageprocessing apparatus is assumed to be a PC.

Referring to FIG. 1, the image processing apparatus 100 includes acentral processing unit (CPU) 101, a read-only memory (ROM) 102, arandom access memory (RAM) 103, a hard disk drive (HDD) 104, a display105, a keyboard 106, a mouse 107, and a data communication unit 108.These elements are mutually interconnected via a data bus 109. Acomputer of the image processing apparatus 100 is configured with, forexample, the CPU 101, ROM 102, and the RAM 103.

The CPU 101, which is a central arithmetic device or a processor, is asystem control unit that controls the entire image processing apparatus100. Moreover, the CPU 101 performs an image processing method describedin the first exemplary embodiment according to a program. Furthermore,while, in FIG. 1, only one CPU is illustrated, the CPU 101 is notlimited to this, but can include a plurality of CPUs.

Programs and an operating system (OS) which are executed by the CPU 101are stored in the ROM 102. The RAM 103 provides a memory for temporarilystoring various pieces of information to be used when programs areexecuted by the CPU 101. The HDD 104 is a storage medium for storing,for example, a database which retains image files and results ofprocessing, such as image analysis. In the first exemplary embodiment,the RAM 103 stores an album creation application, details of which aredescribed below.

The display 105 (display unit) is a device which displays a userinterface (UI) in the first exemplary embodiment or a layout result ofimages to the user. The display 105 can be equipped with a touch sensorfunction. The keyboard 106 is one of input devices, and is used, forexample, to input predetermined information onto a UI displayed on thedisplay 105. The predetermined information is, for example, informationindicating the number of spreads or the number of pages of an albumintended to be created. The mouse 107 is one of input devices, and isused, for example, to click a button on the UI displayed on the display105. Furthermore, the album creation application is activated, forexample, by the user operating the mouse 107 to double-click an iconcorresponding to the album creation application, which is displayed onthe display 105.

The data communication unit 108 (communication unit) is a deviceprovided to communicate with an external apparatus, such as a printer ora server. For example, data generated by the album creation applicationis transmitted via the data communication unit 108 to a printer (notillustrated) or a server (not illustrated) connected to the imageprocessing apparatus 100. Moreover, the data communication unit 108receives still image data present on a server (not illustrated) or asocial networking service (SNS) server (not illustrated). Furthermore,while, in the first exemplary embodiment, the data communication unit108 receives still image data from an SNS server, but the datacommunication unit 108 can also receive moving image data.

The data bus 109 connects the above-mentioned units (102 to 108) to theCPU 101.

In the first exemplary embodiment, the album creation application isconfigured to have the function of generating, at once, a plurality ofpieces of album data including a common page and an individual page. Thecommon page refers to a page which is included in common in all of thepieces of album data generated at once by the album creationapplication. The individual page refers to a page which is included inany one of the pieces of album data generated at once by the albumcreation application and which is not included in common in all of thepieces of album data generated at once by the album creationapplication. Since individual pages indicate contents corresponding torespective pieces of album data, the contents of individual pages differfor respective pieces of album data. Then, in the first exemplaryembodiment, a factor (criterion) for selecting image data included in acommon page and a factor for selecting image data included in anindividual page are different from each other. Then, a factor forselecting image data included in an individual page differs for eachpiece of album data.

For example, suppose that there is a use case where a plurality of usersbrings respective pieces of image data to generate album data. In thiscase, it is requested that a plurality of pieces of album data which isoutput to respective ones of a plurality of users includes a commonpage, which is a page common to the plurality of pieces of album data,and an individual page, which is a page not common to the plurality ofpieces of album data and different for each album. The method forgenerating album data including both a common page and an individualpage includes, for example, a method in which, after album dataconfigured with only common pages is generated, the user manually editsthe album data. However, this method is troublesome for the user and is,therefore, low in usability. As in the first exemplary embodiment,selecting image data included in a common page and image data includedin an individual page according to respective different criteria togenerate a plurality of pieces of album data enables easily generatingalbum data with an appropriate content.

When the album creation application in the first exemplary embodiment isinstalled in the image processing apparatus 100, a start-up icon isgenerated on the top screen (desktop), which is displayed on the display105, by the OS running on the image processing apparatus 100. Then, whenthe user double-clicks the start-up icon with an operation of the mouse107, a program of the album creation application stored in the HDD 104is loaded onto the RAM 103. Then, the program loaded on the RAM 103 isexecuted by the CPU 101, so that the album creation application isactivated.

FIG. 3 is a diagram illustrating an example of a setting screen 301which the album creation application as activated provides. The settingscreen 301 is displayed on the display 105. In response to the usersetting a creation condition for albums via the setting screen 301, thealbum creation application acquires the setting contents designated bythe user.

In the first exemplary embodiment, an image data group for use inalbums, the total number of pieces of spread data, a commercial materialwhich is used for creation of an album, the number of pieces of albumdata which are created at once, and a main subject for each album areable to be set as the creation condition for albums. Furthermore, in thefirst exemplary embodiment, pages include a spread page (hereinafterreferred to as a “spread”) and an individual page. Then, in the firstexemplary embodiment, a spread is equivalent to a pair of individualpages adjacent to each other printed on different sheets, and spreaddata is data which is used to determine a layout of a spread and printthe spread with the determined layout. Furthermore, while, in the firstexemplary embodiment, a configuration in which processing is performedfor each piece of spread data is employed, a configuration in whichprocessing is performed for each piece of page data corresponding to onepage (one individual page) can be employed. Moreover, the main subjectrefers to a subject which is preferentially laid out in an album. Forexample, a particular individual or a category or object of a thing,such as a dog, cat, flower, or food, is able to be set as the mainsubject.

A path box 302 on the setting screen 301 represents a storage location(path) in the HDD 104 for an image data group targeted for albumcreation. When a folder selection button 303 is clicked by the mouse 107being operated by the user, a folder including an image data grouptargeted for album creation is displayed in a tree structure in such amanner as to be able to be selected by the user. Then, a folder pathincluding an image data group selected by the user is displayed in thepath box 302. The image data group designated here is acquired, so thatalbum data is generated. Furthermore, an image data group can beacquired from a storage region, such as a server on a network or an SNSserver, via the data communication unit 108. The image data group asused herein represents candidates for image data for use in albumcreation. Moreover, in designating an image data group targeted foralbum creation, information other than a storage location for the imagedata group targeted for album creation can be input. For example, thedate and time at which an image data group targeted for album creationwas generated (photographs corresponding to the image data group weretaken) (hereinafter referred to as an “image capturing date and time”)can be designated.

Image data targeted for album creation is, for example, still image dataor clipped image data obtained by clipping a frame from moving imagedata. Still image data or clipped image data is data acquired from animaging device. The imaging device can be included in the imageprocessing apparatus 100 or can be included in an external apparatus(for example, a PC, a smartphone, a digital camera, or a tabletterminal) which is an apparatus external to the image processingapparatus 100. Furthermore, when acquiring image data from an externalapparatus, the image processing apparatus 100 acquires the image datavia the data communication unit 108. Moreover, the image processingapparatus 100 can acquire still image data or clipped image data from anetwork or a server via the data communication unit 108. The CPU 101analyzes data appended to image data and determines from where eachpiece of image data was acquired. Furthermore, when acquiring image datafrom an SNS server via the album creation application, the CPU 101 canuse the album creation application to manage from where the image datawas acquired. Furthermore, image data which is used by the albumcreation application is not limited to the above-mentioned type of imagedata, but can be another type of image data.

A number of spreads box 304 is a box used to receive, from the user,designation of the total number of spreads in one album. The user isallowed to designate the total number of spreads by, for example,directly entering numerals in the number of spreads box 304 with use ofthe keyboard 106 or entering numerals in the number of spreads box 304from a list with use of the mouse 107. Furthermore, while the number ofspreads designated here is applied to all of the albums corresponding tothe number designated in a number of albums box 310, for example, aconfiguration in which the total number of spreads is designated foreach album can be employed.

The number of albums box 310 is a box used to receive, from the user,designation of the number of albums which are to be created at once bythe album creation application. The user is allowed to designate thenumber of albums which are to be created by the album creationapplication by, for example, directly entering numerals in the number ofalbums box 310 with use of the keyboard 106 or entering numerals in thenumber of albums box 310 from a list with use of the mouse 107.

Examples of spreads included in albums which are generated by the albumcreation application are described with reference to FIG. 2. Asmentioned above, in the first exemplary embodiment, since a common pageand an individual page are included in each of albums the number ofwhich is designated via the number of albums box 310, respective albumsdiffer from each other in the content of at least a part thereof.

Here, suppose that three albums (albums 1 to 3) are created at a timewith the number of albums designated at the number of albums box 310. Animage 201 is an example of an image arranged in a spread of the album 1,and letters in the image 201 represent main subjects shown in the image201. A spread 202 is a spread composed of a common page and is includedin all of the albums 1 to 3. Then, in this way, the common page isincluded in all of the albums and is, therefore, laid out with a focuson an image in which the main subjects designated in the respectivealbums are shown. On the other hand, each of spreads 203, 204, and 205is a spread composed of an individual page. Therefore, each of spreads203, 204, and 205 is included in only one album. Then, for example, thespread 203, which is included in the album 1, is laid out with a focuson an image in which a main subject A, which is a main subjectdesignated in the album 1, is shown. Moreover, the spread 204, which isincluded in the album 2, is laid out with a focus on an image in which amain subject B, which is a main subject designated in the album 2, isshown. In this way, in the first exemplary embodiment, a plurality ofalbums each of which is composed of two types of spreads createdaccording to respective different criteria is created.

A main subject designation portion 305 is a region used to receive, fromthe user, designation of a main subject with respect to each of thealbums corresponding to the number entered in the number of albums box310. For example, the user is able to designate, with the mouse 107, amain subject for every album by dragging and dropping image data inwhich a main subject is shown to a region corresponding to each albumincluded in the main subject designation portion 305.

A number of spread photographs portion 306 is a region used to receive,from the user, designation of the largeness or smallness of the numberof pieces of image data which are arranged in each spread of an album tobe created. The user is able to increase the number of pieces of imagedata which are arranged in each spread, by setting a slider bar to the“large” side. Furthermore, as the number of pieces of image data whichare arranged in each spread is increased, the number of pieces of imagedata which are arranged in each album is increased accordingly.Moreover, the user is able to decrease the number of pieces of imagedata which are arranged in each spread, by setting the slider bar to the“small” side. Furthermore, as the number of pieces of image data whichare arranged in each spread is decreased, the number of pieces of imagedata which are arranged in each album is decreased accordingly.Furthermore, while the largeness or smallness of the number of pieces ofimage data designated here is applied to all of the albums correspondingto the number designated in the number of albums box 310, for example, aconfiguration in which the largeness or smallness of the number ofpieces of image data is designated for each album can be employed.

A commercial material designation portion 307 is a region used toreceive, from the user, designation of a commercial material for use inan album to be created. The user is able to perform entry in thecommercial material designation portion 307 to set, for example, thesize of an album, the type of paper for the album, the type of a coversheet of the album, and the type of a biding portion of the album.

An OK button 308 is a button used to receive, from the user, an inputindicating that the setting for an album to be created is completed.When the OK button 308 is clicked by the user via the mouse 107, thealbum creation application acquires the contents which have been set onthe setting screen 301, and then uses the acquired contents for albumdata creation processing described below.

A reset button 309 is a button used to reset the respective pieces ofsetting information on the setting screen 301.

Furthermore, the setting screen 301 can be configured to allow settingsother than the above-mentioned settings to be performed. For example,the setting screen 301 can be configured to allow, for example, settingconcerning a moving image and setting of the acquisition destination ofstill or moving image data, or can be configured to allow the mode of analbum targeted for creation to be designated. Furthermore, the mode ofan album is a mode according to which to preferentially lay out an imagecontaining a predetermined object in a template, and includes, forexample, three modes of “person”, “animal”, and “food”. Furthermore, themode of an album can be reworded as, for example, the theme of an album.For example, in a case where the mode “person” is selected, an imagecontaining a person is preferentially laid out in the template.Moreover, in a case where the mode “animal” is selected, an imagecontaining an animal is preferentially laid out in the template.Moreover, in a case where the mode “food” is selected, an imagecontaining food is preferentially laid out in the template. Furthermore,there can be a mode for preferentially laying out image datarepresenting an image containing an object other than theabove-mentioned three objects in the template. Moreover, a plurality ofmodes can be concurrently selected. In that case, an image containing atleast one of a plurality of objects corresponding to the selectedplurality of modes is preferentially laid out in the template. The useris allowed to select any one icon of the plurality of iconscorresponding to the respective modes by clicking with the mouse 107.Then, a mode corresponding to the selected icon is set as the mode of analbum to be created. Furthermore, the modes of albums are not limited tothe above-mentioned three modes, but can additionally include othermodes, such as “building”, “vehicle”, and “flower”.

FIGS. 4A, 4B, and 4C are flowcharts illustrating automatic layoutprocessing which is performed by the album creation applicationaccording to the first exemplary embodiment. The flowcharts illustratedin FIGS. 4A to 4C are implemented, for example, by the CPU 101 loading aprogram corresponding to the album creation application stored in theHDD 104 onto the ROM 102 or the RAM 103 and executing the program.Furthermore, in the first exemplary embodiment, the album creationapplication, which is stored in the HDD 104, is activated by an iconcorresponding to the album creation application displayed on the display105 being operated by the user with the mouse 107, and then performs thefollowing processing.

First, in step S401, the CPU 101 sets an album creation condition.Specifically, for example, the CPU 101 receives setting of the albumcreation condition from the user via the screen illustrated in FIG. 3.Details of the setting of the album creation condition are as describedabove with reference to FIG. 3.

In step S402, the CPU 101 acquires image data included in an image datagroup designated in the path box 302. Specifically, the CPU 101 reads animage data group designated in the path box 302 from the HDD 104 andthen loads image data included in the image data group onto the RAM 103.

In step S403, the CPU 101 performs image analysis processing foranalyzing image data loaded onto the RAM 103 in step S402. FIG. 4B is aflowchart illustrating image analysis processing which is performed bythe album creation application according to the first exemplaryembodiment.

In step S40301, the CPU 101 acquires information about the imagecapturing date and time from information (for example, Exif information)accompanying the image data acquired in step S402.

In step S40302, the CPU 101 acquires information about a feature amountof image data from the image data acquired in step S402. Examples of thefeature amount include focus. Examples of the method for detecting anedge to acquire the feature amount concerning focus include using acommonly-used Sobel filter. The gradient of an edge, in other words,luminance gradient, can be calculated by detecting the edge via theSobel filter and dividing a luminance difference between the startingpoint and end point of the edge by a distance between the starting pointand end point thereof. The result obtained by calculating an averagegradient of edges in the image enables deeming an image with a largeaverage gradient to be more in focus than an image with a small averagegradient. Then, setting a plurality of threshold values different fromeach other with respect to the gradient enables outputting an evaluationvalue of the amount of focusing by determining which threshold value thegradient is equal to or greater than. In the first exemplary embodiment,two different threshold values are previously set, and the amount offocusing is determined in three levels of “∘”, “Δ”, and “×”. Thethreshold values are previously set, for example, through experiment insuch a manner that the gradient of focus desired to be employed for analbum is set to level “∘”, the allowable gradient of focus is set tolevel “Δ”, and the unallowable gradient of focus is set to level “×”.Furthermore, for example, the setting of threshold values can beprovided by, for example, a creator of the album creation application,or can be configured to be able to be performed on a user interface.

In step S40303, the CPU 101 performs object detection processing on eachpiece of image data acquired in step S402. First, the CPU 101 detectsthe face of a person from an image represented by each piece of imagedata acquired in step S402. A known method can be used for processingfor face detection, and, for example, AdaBoost, in which a strongclassifier is generated from a plurality of previously-prepared weakclassifiers, is used. In the first exemplary embodiment, the face imageof a person (object) is detected by a strong classifier generatedaccording to AdaBoost. Moreover, the CPU 101 not only extracts the faceimage but also acquires the upper left coordinate values and lower rightcoordinate values of the position of the detected face image. Acquiringsuch two types of coordinates enables the CPU 101 to acquire theposition of the face image and the size of the face image. Moreover, instep S40303, the CPU 101 can generate a strong classifier according toAdaBoost with not only a face but also another object, such as ananimal, for example, a dog or cat, a flower, food, a building, astationary article, or a vehicle, used as a detection target. This alsoenables the CPU 101 to detect objects other than faces. Moreover, theCPU 101 can search for an object with use of not AdaBoost but, forexample, a deep neural network.

In step S40304, the CPU 101 performs personal recognition processing.Specifically, the CPU 101 performs personal recognition by making acomparison in similarity between the face image extracted in step S40303and a representative face image stored for each personal ID in a facedictionary database. The CPU 101 determines the similarity between theface image extracted in step S40303 and each representative face image,and sets the ID of a representative face image with the similarity equalto or greater than a threshold value and with the highest similarity asthe ID of the face image extracted in step S40303. In other words, theCPU 101 identifies a person corresponding to the representative faceimage with the similarity equal to or greater than a threshold value andwith the highest similarity as a person corresponding to the face imageextracted in step S40303. Furthermore, in a case where the similaritybetween the face image extracted in step S40303 and every representativeface image stored in the face dictionary database is less than thethreshold value, the CPU 101 assigns a new personal ID to the extractedface image and registers the extracted face image as a new person withthe face dictionary database. Furthermore, the face dictionary databaseis stored in, for example, the HDD 104.

In step S404, the CPU 101 determines whether the image analysisprocessing has been completed with respect to all of the pieces of imagedata stored in the image data group designated in the path box 302. Ifit is determined that the image analysis processing has not yet beencompleted (NO in step S404), the CPU 101 returns the processing to stepS402, in which the CPU 101 acquires image data that is not yet subjectedto the image analysis processing. If it is determined that the imageanalysis processing has been completed (YES in step S404), the CPU 101advances the processing to step S405.

In step S405, the CPU 101 performs scene segmentation. The scenesegmentation refers to segmenting all of the pieces of image dataacquired in steps S401 to S404 by scenes and managing the segmentedpieces of image data as a plurality of image groups. More specifically,the CPU 101 segments all of the pieces of image data into a plurality ofimage groups based on time differences in image capturing date and timeof respective pieces of image data calculated from the image capturingdate and time information acquired in step S403. Furthermore, in thefollowing description, each image group obtained by segmenting all ofthe pieces of image data (main image group) is referred to as a“sub-image group”. In the first exemplary embodiment, for example, theCPU 101 performs scene segmentation according to the following method.First, if the difference between the image capturing date and time ofthe N-th captured image data and the image capturing date and time ofthe (N+1)-th captured image data in ascending order of the imagecapturing date and time is equal to or less than one day, the CPU 101performs the following operation. If the difference between the imagecapturing date and time of the N-th captured image data and the imagecapturing date and time of the (N+1)-th captured image data in ascendingorder of the image capturing date and time is equal to or greater thansixteen hours, the CPU 101 sets a boundary of groups between the N-thcaptured image data and the (N+1)-th captured image data. Moreover,there may be a case where the difference in image capturing date andtime between pieces of captured image data adjacent in ascending orderof the image capturing date and time is less than sixteen hours betweenevery piece of captured image data. In this case, if the differencebetween the image capturing date and time of the first captured imagedata and the image capturing date and time of the last captured imagedata in a captured image data group captured in one day is less thanfour hours, the CPU 101 sets the captured image data group captured inthat day as one sub-image group. On the other hand, there may be a casewhere the difference between the image capturing date and time of thefirst captured image data and the image capturing date and time of thelast captured image data in the captured image data group captured inthat day is equal to or greater than four hours. In this case, if thenumber of pieces of captured image data captured in that day is lessthan 50 (the number of captured images captured in one day is less than50), the CPU 101 sets the captured image data group captured in that dayas one sub-image group. FIG. 5A illustrates an example of such groupingof pieces of captured image data. In FIGS. 5A to 5C, the horizontal axisindicates image capturing date and time (the more left side being oldand the more right side being new), and the vertical axis indicates thenumber of pieces of captured image data. Referring to FIG. 5A, thecaptured image data group is segmented into eight sub-image groups,including groups 601 to 608. In FIG. 5A, each arrow indicates a boundaryof groups. Furthermore, the threshold value regarding the difference inimage capturing date and time and the threshold value regarding thenumber of pieces of captured image data used in the above description ofthe scene segmentation method are merely examples, and are not limitedto the above-mentioned values. Furthermore, an image group obtained bysegmentation is referred to as a “segmented image group”, and onesegmented image group is set with respect to every scene.

Moreover, while, in the first exemplary embodiment, image capturing dateand time information is used for scene segmentation, the first exemplaryembodiment is not limited to this. For example, using image capturingposition information, the CPU 101 can set two pieces of image data closein image capturing position as one segmented image group. Moreover, forexample, the CPU 101 can use a personal recognition result. For example,the CPU 101 previously registers pieces of image data about studentsbelonging to a given group (for example, a class or a club) with analbum creation application. Then, the CPU 101 can use personalrecognition results obtained by recognition with the registered piecesof image data to set two pieces of image data in which the studentsbelonging to the group are shown as one segmented image group. Moreover,for example, the CPU 101 can use tag information appended with, forexample, the name of a folder in which images are stored or with asocial networking service. Specifically, for example, the CPU 101 canset two pieces of image data having tag information including a searchword, such as “a school athletic meet” or “school trip”. The search wordis, for example, input from the user via the setting screen 301.

In step S406, the CPU 101 performs scene classification. Specifically,the CPU 101 performs scoring on the sub-image groups obtained byperforming scene segmentation in step S407 by every type of scene, andclassifies the sub-image groups into the respective types of scenes withthe highest score. Furthermore, in the following description, thescoring operation in step S406 is referred to as “scene classificationscoring”. In the first exemplary embodiment, the types of scenes areassumed to include “travel”, “daily life”, and “ceremony”, and anexample in which each sub-image group is classified into any one ofthese scenes is described. Furthermore, scene classification scoring isperformed with use of a scene classification table in which informationabout feature amounts corresponding to the respective types of scenes isstored.

Here, a method for generating the scene classification table, which ispreviously stored in a program, is described. First, a plurality ofimage data groups previously determined to be classified into therespective scenes of “travel”, “daily life”, and “ceremony” is collectedby the designer of the program, and feature amounts are acquired withrespect to the collected image data groups. Examples of the featureamounts acquired in this instance include an image capturing period, thenumber of captured images, and the number of persons subjected to imagecapturing. The image capturing period is an image capturing timedifference between the first generated image data and the last generatedimage data in each image data group. The number of captured images isthe number of images in each image data group. The number of personssubjected to image capturing is the average number of faces contained ineach image represented by each image data group. As a result, forexample, with respect to one image data group including a plurality ofpieces of image data previously determined to be a travel scene, thefeature amounts, which include an image capturing period, the number ofcaptured images, and the number of persons subjected to image capturing,are acquired. Furthermore, the feature amounts are not limited to animage capturing period, the number of captured images, and the number ofpersons subjected to image capturing, and other feature amounts can alsobe used or only any one of these feature amounts can also be used.

Then, the acquisition of feature amounts, such as the above-mentionedimage capturing period, the number of captured images, and the number ofpersons subjected to image capturing, is also performed with respect topreviously-collected other image data groups. Next, an average value anda standard deviation of the image capturing period, an average value anda standard deviation of the number of captured images, and an averagevalue and a standard deviation of the number of persons subjected toimage capturing per one image are obtained based on the feature amountsacquired from a plurality of image data groups collected for each scene.The values obtained in this way and the respective types of scenes areassociated with each other, so that a scene classification table whichindicates the average value and standard deviation for each type ofscene is generated.

In the first exemplary embodiment, a table 600 illustrated in FIG. 6 isassumed to be used as the scene classification table. In the table 600,an average value and a standard deviation of each of an image capturingperiod 602, the number of captured images 603, and the number of personssubjected to image capturing 604 are previously registered inassociation with a scene ID 601. The CPU 101 previously stores the table600 in a storage region, such as the ROM 102 or the HDD 104, prior toscene classification processing. Furthermore, the table 600 can bepreviously incorporated in the program of the album creationapplication, or can be previously incorporated in a program outside thealbum creation application.

Here, the scene classification scoring is described. After the albumcreation application is activated, the CPU 101 calculates the scores ofthe respective feature amounts of the image capturing period, the numberof captured images, and the number of persons subjected to imagecapturing with respect to each of the sub-image groups obtained byscene-segmenting, in step S405, the image data group designated by theuser via the path box 302. The CPU 101 calculates the scores and theiraverage scores of the image capturing period, the number of capturedimages, and the number of persons subjected to image capturing withrespect to each sub-image group using the average value and standarddeviation for each scene illustrated in FIG. 6 according to thefollowing formulae (1) and (2). Furthermore, the average value of apredetermined feature amount corresponding to a predetermined type ofscene and the standard deviation of the predetermined feature amountcorresponding to the predetermined type of scene in the followingformulae are registered with the table 600. Moreover, the featureamounts are acquired with respect to three feature items, i.e., theimage capturing period, the number of captured images, and the number ofpersons subjected to image capturing. Therefore, the number of featureitems in this example is three.

First, the CPU 101 calculates the scores of the sub-image groups for therespective types of scenes and the respective feature amounts accordingto formula (1).

The score of a predetermined feature amount corresponding to apredetermined type of scene in a sub-image group=50−|10×(the averagevalue of the predetermined feature amount corresponding to thepredetermined type of scene−the predetermined feature amount in thesub-image group)/the standard deviation of the predetermined featureamount corresponding to the predetermined type of scene|  (1)

With this, for example, in a sub-image group, the respective scores ofthe image capturing period, the number of captured images, and thenumber of persons subjected to image capturing corresponding to the“travel” scene are acquired. Furthermore, the CPU 101 acquires thescores with respect to not only the “travel” scene but also the othertypes of scenes.

Next, the CPU 101 averages, according to formula (2), the respectivescores obtained according to formula (1), thus acquiring the scoreconcerning each type of scene in the sub-image group.

The score concerning a predetermined type of scene in a sub-imagegroup=(the score of the image capturing period+the score of the numberof captured images+the score of the number of persons subjected to imagecapturing corresponding to the predetermined type of scene in thesub-image group)/the number of feature items   (2)

With this, for example, in a sub-image group, the scores concerning therespective types of scenes of “travel”, “daily life”, and “ceremony” areacquired. Furthermore, the CPU 101 performs the scene classificationscoring with respect to every sub-image group.

In the above-described way, the respective scores for “travel”, “dailylife”, and “ceremony” are acquired with respect to each sub-image group.Then, the CPU 101 classifies a scene with the score ranking first as thescene of the corresponding sub-image group. For example, a sub-imagegroup 5 corresponding to a scene 5 obtained by scene segmentationillustrated in FIG. 5A is assumed to have the image capturing periodbeing 36 hours, the number of captured images being 300, and the numberof persons subjected to image capturing being 1.7. Then, in thesub-image group 5, the score concerning “travel” becomes 45.32, thescore concerning “daily life” becomes 18.38, and the score concerning“ceremony” becomes −29.92, so that the sub-image group 5 is classifiedas “travel”, which is a scene with the highest score. Furthermore, whichsub-image group is classified as which scene is managed with a scene IDin a discriminable manner Moreover, in a case where there are the samescores among scores concerning the respective scenes, the CPU 101classifies a sub-image group as any one of the scenes with the samescore according to predetermined ordering of preference. Specifically,the CPU 101 classifies a sub-image group as a scene which is higher inpreference among the scenes with the same score. Furthermore, forexample, in the first exemplary embodiment, the ordering of preferenceis assumed to be the order of “daily life”>“ceremony”>“travel”. Theordering of preference is not limited to this order, but, for example,the ordering of preference can be configured to be settable by the user.

Furthermore, while the feature amounts for use in scene classificationscoring are acquired with respect to three feature items, i.e., theimage capturing period, the number of captured images, and the number ofpersons subjected to image capturing, the first exemplary embodiment isnot limited to this. For example, a flag indicating whether apredetermined object corresponding to each scene is included in imagedata can be set as a feature amount. For example, in a case where apredetermined object corresponding to “travel” is “sky”, the scoreconcerning “travel” is set in such a manner as to become high withrespect to a sub-image group in which the number of pieces of image dataincluding “sky” is large. Moreover, personal identification informationcan be set as a feature amount. For example, suppose that there arescenes of “wedding ceremony” and “wedding reception”, into which a sceneof “ceremony” is further finely classified. Then, the CPU 101identifies, as the bride and groom, two persons who are most frequentlyincluded in pieces of image data in a main image group. Then, a flagindicating whether only the bride and groom are included or whether anyperson other than the bride and groom is included is set as a featureamount. In this case, for example, the score concerning “weddingceremony” is set in such a manner as to become high with respect to asub-image group in which the number of pieces of image data includingonly the bride and groom is large, and the score concerning “weddingreception” is set in such a manner as to become high with respect to asub-image group in which the number of pieces of image data includingany person other than the bride and groom is large. Furthermore, sceneclassification can be performed with, for example, outputting obtainedby machine learning using a convolutional neural network (CNN).

In step S407, the CPU 101 sets a main subject with respect to therespective albums corresponding to the number entered in the number ofalbums box 310 according to an input from the user to the main subjectdesignation portion 305. Here, for the purpose of illustration, supposethat a main subject 1 is set for the album 1 and a main subject 2 is setfor the album 2.

In step S408, the CPU 101 performs image scoring. The image scoring instep S408 refers to assigning a score (scoring) evaluated from aviewpoint described below for each piece of image data, and is referredto at the time of selecting image data representing an image to beplaced in a template as described below. In the first exemplaryembodiment, image scoring is performed in a plurality of viewpointsbased on the feature amount of each piece of image data acquired in stepS40302. In other words, in the first exemplary embodiment, a pluralityof scores is assigned to each piece of image data by image scoring.Specifically, in the first exemplary embodiment, scores in threeviewpoints, i.e., a score that is based on image quality, a score thatis based on the main subject 1, and a score that is based on the mainsubject 2, are assigned to each piece of image data.

First, image scoring that is based on image quality is described. In thefirst exemplary embodiment, for example, in image scoring that is basedon image quality, a feature amount of, for example, focus or the numberof pixels is used. Specifically, for example, a score that is based onimage quality is assigned in such a manner that a score assigned toimage data in which the feature amount of focus acquired in step S40302is at level “∘” becomes higher than a score assigned to image data inwhich the feature amount of focus acquired therein is at level “Δ” or“×”. Moreover, for example, a score that is based on image quality isassigned to each piece of image data in such a manner that a scoreassigned to image data representing an image having a large number ofpixels becomes higher than a score assigned to image data representingan image having a small number of pixels. Furthermore, a feature amountfor use in image scoring that is based on image quality can be imagecapturing information such as lens information used at the time of imagecapturing, or can be a compression format of images to be input to anapplication.

Next, image scoring that is based on the main subject 1 and imagescoring that is based on the main subject 2 are described. These imagescoring operations are performed based on information about whether amain subject, who is identified with use of the size of a face analyzedin step S40303 and personal recognition performed in step S40304, isincluded in in image data. For example, a score that is based on themain subject 1 is assigned to image data in such a manner that a scoreassigned to image data in which the main subject 1 is included becomeshigher than a score assigned to image data in which the main subject 1is not included. Moreover, for example, a score that is based on themain subject 1 is assigned to image data in such a manner that a scoreassigned to image data in which the area (size) occupied by the mainsubject 1 is large becomes higher than a score assigned to image data inwhich the area (size) occupied by the main subject 1 is small. Moreover,for example, a score that is based on the main subject 1 is assigned toimage data in such a manner that a score assigned to image data in whicha smiling subject 1 is included becomes higher than a score assigned toimage data in which an unsmiling subject 1 is included. With regard tothe subject 2, image scoring is also performed in the same viewpoint.

In step S409, the CPU 101 determines whether the image scoring has beencompleted with respect to all of the pieces of image data included inthe image data group designated in the path box 302. If it is determinedthat the image scoring has not yet been completed (NO in step S409), theCPU 101 returns the processing to step S408, in which the CPU 101performs image scoring on image data that is not yet subjected to theimage scoring. If it is determined that the image scoring has beencompleted (YES in step S409), the CPU 101 advances the processing tostep S410.

In step S410, the CPU 101 specifies a score for common page and a scorefor individual page with respect to all of the pieces of image dataacquired in step S402 in the light of a result of image scoringperformed in step S408. The common page is a page included in common inall of the albums designated to be created in the setting screen 301. Onthe other hand, the individual page is a page included in only one albumout of the albums designated to be created in the setting screen 301.The score for individual page is specified for each of the albumsdesignated to be created in the setting screen 301.

A score ^(I)score_(s) for common page of image data I is expressed bythe following formula (3).

^(I)score_(s)=Σ_(i) w _(i)·^(I)score_(i)   (3)

Here, ^(I)score_(i) denotes the i-th score out of the respective scores(a score that is based on image quality, a score that is based on themain subject 1, and a score that is based on the main subject 2)assigned to the image data I, and w_(i) denotes a weight added to eachscore. In the first exemplary embodiment, the weight w_(i) is set equalto ⅓, so that the score that is based on image quality, the score thatis based on the main subject 1, and the score that is based on the mainsubject 2 are evenly evaluated. With this, the score for common page isspecified in such a manner that the score of an image the image qualityof which is good and in which the number of main subjects shown therein(or the size of each main subject shown therein) is large becomes high.Thus, the score for common page is specified in consideration of eachmain subject corresponding to each of all of the pieces of album datadesignated to be created.

A score ^(I)score_(IO) for individual page corresponding to album data Oof the image data I is expressed by the following formula (4).Furthermore, O denotes an album number allocated to each of a pluralityof pieces of album data designated by the user to be created. Thus, inthe first exemplary embodiment, scores for individual page the number ofwhich corresponds to the number of pieces of album data are assigned tothe image data I.

$\begin{matrix}{{{}_{}^{}{}_{}^{}} = \left\{ \begin{matrix}0 & \left( {{{}_{}^{}{}_{}^{}} = 0} \right) \\\frac{{w_{q} \cdot {{}_{}^{}{}_{}^{}}} + {w_{0} \cdot {{}_{}^{}{}_{}^{}}}}{2} & ({otherwise})\end{matrix} \right.} & (4)\end{matrix}$

Here, ^(I)score_(q) denotes a score that is based on image quality,assigned to the image data I, ^(I)score_(o) denotes a score that isbased on a main subject _(s)O corresponding to album data O assigned tothe image data I, w_(q) denotes a weight added to a score that is basedon image quality, and w_(o) denotes a weight added to a score that isbased on the main subject s^(O). In the first exemplary embodiment, theweights w_(q) and w_(o) are set equal to 1, so that the score forindividual page is specified in such a manner that the score of an imagethe image quality of which is good and in which the number of mainsubjects _(s)O shown therein (or the size of each main subject _(s)Oshown therein) is large becomes high. In other words, the score forindividual page is specified without consideration of main subjectsother than the main subject _(s)O (main subjects not corresponding tothe album data O). Specifying a score for individual page is performedfor each piece of album data. Thus, as a result, in step S410, scoresfor individual page the number of which corresponds to the number ofpieces of album data designated to be created in the setting screen 301are specified, and one score for common page is specified. Furthermore,in the first exemplary embodiment, in a case where the score^(I)score_(o) is 0, the score ^(I)score_(IO) becomes 0 irrespective ofthe score ^(I)score_(q).

In this way, in the first exemplary embodiment, an evaluation for commonpage and an evaluation for individual page are performed based on therespective different factors. Specifically, an evaluation for commonpage is performed based on factors about a plurality of main subjectsrespectively corresponding to a plurality of pieces of album data. Onthe other hand, an evaluation for individual page corresponding to analbum O is performed based on a factor about a main subjectcorresponding to the album O, and is not performed based on a factorabout a main subject corresponding to another album. This enablesappropriately performing an estimation for common page and an estimationfor individual page.

Furthermore, while specifying a score for common page and specifying ascore for individual page are performed based on the image quality andmain subjects, the first exemplary embodiment is not limited to this.For example, with regard to image data in which no main subject isincluded (for example, image data classified into “landscape”), a scorethereof can be specified based on another piece of image informationassociated with the image data. Moreover, for example, image data whichis small in the difference in image capturing time from image datahaving a high score can be controlled to have a high score. This enablesalso assigning a high score to, for example, image data which is likelyto have relevance to image data in which a main subject is included.

In step S411, the CPU 101 generates spread data representing a spreadincluded in an album with use of the scores specified in step S410. FIG.4C is a flowchart illustrating spread data generation processing whichis performed by the album creation application according to the firstexemplary embodiment.

In step S41101, the CPU 101 reads one score out of the scores specifiedin step S410 with respect to all of the pieces of image data acquired instep S402. In the first exemplary embodiment, suppose that the number ofalbums designated by the user to be created is two. Then, suppose thatthe CPU 101 reads a score of each piece of image data in the order of ascore for individual page about the album 1, a score for individual pageabout the album 2, and a score for common page. However, the firstexemplary embodiment is not limited to this, and the CPU 101 can readany score in first. Furthermore, in a case where a score for individualpage is read in step S41101, in the subsequent processing, the CPU 101generates spread data for individual page, and, in a case where a scorefor common page is read in step S41101, in the subsequent processing,the CPU 101 generates spread data for common page.

In step S41102, the CPU 101 acquires candidate image data, which servesas a candidate for image data to be arranged in spreads, from all of thepieces of image data acquired in step S402, based on the score read instep S41101. Furthermore, in a case where the score read in step S41101is a score for individual page, here, candidate image data serving as acandidate for image data to be arranged in a template for individualpage is acquired. Moreover, in a case where the score read in stepS41101 is a score for common page, here, candidate image data serving asa candidate for image data to be arranged in a template for common pageis acquired. In the first exemplary embodiment, the CPU 101 acquires, ascandidate image data, image data having a score higher than or equal toa threshold value as the score read in step S41101. In a case wherethere is no image data having a score higher than or equal to thethreshold value, the CPU 101 acquires, as candidate image data, imagedata having the highest score. Furthermore, while, here, the CPU 101acquires candidate image data from all of the image data groups, thefirst exemplary embodiment is not limited to this. Image data previouslyselected in image selection processing described below can be excludedfrom targets for acquisition.

In step S41103, the CPU 101 sets the number of spreads based on a resultof the scene segmentation performed in step S405 and the candidate imagedata acquired in step S41102. First, the CPU 101 specifies how manypieces of candidate image data belong to each scene. In the firstexemplary embodiment, spread data is generated for each scene.Therefore, the CPU 101 specifies the number of pieces of spread datacorresponding to each scene according to formula (5). Furthermore, themaximum number of pieces of image data per spread is determinedaccording to an input entered in the number of spread photographsportion 306.

In the following description, first, a case where the score read in stepS41101 is a score for individual page is described. Since spread datacorresponding to an individual page is generated from an aggregation ofpieces of spread data corresponding to each scene, the CPU 101 specifiesthe number of pieces of spread data corresponding to individual pagesaccording to formula (6). S denotes a scene number allocated to eachscene.

Number of pieces of spread data of scene S=ceil (number of pieces ofcandidate image data belonging to scene S÷maximum number of pieces ofimage data per spread)   (5)

Number of pieces of spread data corresponding to individual pages=Σ_(s)number of pieces of spread data of scene S (6)

Furthermore, in a case where the number of pieces of spread datacorresponding to individual pages specified in formula (6) is largerthan or equal to the number (total number of pieces of spread data)designated by the user in the number of spreads box 304, the CPU 101re-specifies the number of pieces of spread data corresponding toindividual pages according to the following formula (7).

Number of pieces of spread data corresponding to individual pages=totalnumber of pieces of spread data−1 (7)

As mentioned above, in the first exemplary embodiment, a common page andan individual page are mixed in an album. To cause a spread composed ofa common page to be included in an album, the above-mentioned specifyingoperation is performed. Furthermore, an upper limit value and a lowerlimit value can be set to the number of pieces of spread datacorresponding to an individual page. Furthermore, in a case where thescore read in step S41101 is a score for individual page about the album1, the number of pieces of spread data corresponding to individual pagesincluded in the album 1 is specified by the above-described processing.Then, in a case where the score read in step S41101 is a score forindividual page about the album 2, the number of pieces of spread datacorresponding to individual pages included in the album 2 is specifiedby the above-described processing.

Next, a case where the score read in step S41101 is a score for commonpage is described. Other than spreads corresponding to individual pagesare spreads corresponding to common pages. Therefore, the CPU 101specifies the number of pieces of spread data corresponding to commonpages according to formula (8).

Number of pieces of spread data corresponding to common pages=totalnumber of pieces of spread data−MIN (number of pieces of spread datacorresponding to individual pages)   (8)

Thus, in the first exemplary embodiment, a value obtained by subtractingthe number of pieces of spread data corresponding to individual pagesfrom the total number of pieces of spread data is the number of piecesof spread data corresponding to common pages.

Furthermore, while, in the first exemplary embodiment, the CPU 101 setsthe number of pieces of spread data corresponding to individual pagesand the number of pieces of spread data corresponding to common pagesaccording to the number of pieces of candidate image data included ineach scene, the first exemplary embodiment is not limited to this. Forexample, each number of pieces of spread data can be a number previouslyset in the album creation application or a number designated by theuser.

In step S41104, the CPU 101 allocates the candidate image data acquiredin step S41102 to each piece of spread data. Here, first, the CPU 101segments the candidate image data acquired in step S41102 into imagedata groups the number of which corresponds to the number of pieces ofspread data specified in step S41103. Furthermore, if the candidateimage data acquired in step S41102 is candidate image data forindividual page, the candidate image is segmented into image data groupsthe number of which corresponds to the number of spreads correspondingto individual pages. Moreover, if the candidate image data acquired instep S41102 is candidate image data for common page, the candidate imageis segmented into image data groups the number of which corresponds tothe number of spreads corresponding to common pages. Specifically, theCPU 101 arranges the candidate image data acquired in step S41102 in theorder of image capturing time based on the information acquired in stepS40301. Then, if a difference in image capturing time between adjacentpieces of image data is larger than a threshold value, the CPU 101segments the candidate image data between the adjacent pieces of imagedata. This method of segmentation is repeated until image data groupsthe number of which corresponds to the number of spreads specified instep S41103 are formed. After that, the CPU 101 allocates the respectiveimage data groups formed by segmentation to the top position of eachspread of the album in order from an image data group which is top inimage capturing date and time. Thus, the CPU 101 allocates image datagroups to each spread of the album in the order of image capturing dateand time. This enables creating an album in which pieces of image dataare arranged in the order of image capturing date and time. Furthermore,within one double-page spread, images do not need to be arranged in theorder of image capturing date and time.

In step S41105, the CPU 101 performs image selection processing forselecting pieces of image data to be arranged in a spread. Here, anexample in which the CPU 101 selects four pieces of image data from asegmented image data group allocated to a certain spread is describedwith reference to FIGS. 7A, 7B, 7C, 7D, 7E, 7F, 7G, 7H, and 7I.Furthermore, the spread is an area for two pages, but each of the firstspread (front cover) and the last spread (back cover) is an area for onepage. Moreover, how many pieces of image data are selected in whichspread data is determined, for example, according to, for example, aninput entered by the user to the number of spread photographs portion306. For example, in a case where the number of photographs that isbased on an input entered by the user to the number of spreadphotographs portion 306 is four, the number of pieces of image datawhich are selected in step S41105 is determined in such a manner thatthe average number of pieces of image data included in each spreadwithin the album becomes four. Therefore, the numbers of pieces of imagedata included in the respective spreads within the album can bedifferent from each other. Moreover, if pieces of image data areselected from a segmented image data group for individual page, theabove-mentioned processing is performed based on a score for individualpage. Image data selected based on a score for individual page from asegmented image data group for individual page is individual image data.Moreover, if pieces of image data are selected from a segmented imagedata group for common page, the above-mentioned processing is performedbased on a score for common page. Image data selected based on a scorefor common page from a segmented image data group for common page iscommon image data.

FIG. 7A illustrates the time difference (segmented image capturingperiod) in the image capturing date and time between image data with theearliest image capturing date and time and image data with the latestimage capturing date and time in a segmented image data group allocatedto a spread, in other words, the image capturing period of the segmentedimage data group. Furthermore, here, selection of pieces of image datais performed in the order of a main slot and a sub-slot. Here, atemplate corresponding to a spread is assumed to contain one main slot.Therefore, image data to be selected as the first piece of image data isimage data for the main slot. The CPU 101 selects, as image data for themain slot, image data (1) with the highest score specified in step S410among pieces of image data corresponding to the segmented imagecapturing period illustrated in FIG. 7B.

Image data to be selected as the second or subsequent piece of imagedata is image data for the sub-slot. Selection of the second orsubsequent piece of image data is performed according to a method suchas that described in the following in such a way as not to concentrateon a part of the segmented image capturing period. First, the CPU 101divides the segmented image capturing period into two periods asillustrated in FIG. 7C. Next, as illustrated in FIG. 7D, the CPU 101selects the second piece of image data from pieces of image datagenerated in a segmented image capturing period (a period indicated by asolid line in FIG. 7D) in which the first piece of image data is notselected. Furthermore, the CPU 101 selects, as the second piece of imagedata, image data (2) with the highest score specified in step S410 amongthe pieces of image data generated in the segmented image capturingperiod in which the first piece of image data is not selected. Next, asillustrated in FIG. 7E, the CPU 101 divides each of the segmented imagecapturing periods illustrated in FIG. 7D into two periods. Then, asillustrated in FIG. 7F, the CPU 101 selects the third piece of imagedata from pieces of image data generated in segmented image capturingperiods (periods indicated by a solid line in FIG. 7F) in which neitherof the first piece of image data and the second piece of image data isselected. Furthermore, the CPU 101 selects, as the third piece of imagedata, image data (3) with the highest score specified in step S410 amongthe pieces of image data generated in the segmented image capturingperiods in which neither of the first piece of image data and the secondpiece of image data is selected. Then, the CPU 101 selects, as thefourth piece of image data, image data with the highest score specifiedin step S410 among pieces of image data generated in a segmented imagecapturing period in which none of the first piece, the second piece, andthe third piece of image data is selected.

Next, an example in which there is no image generated in a segmentedimage capturing period in which none of the first piece, the secondpiece, and the third piece of image data is selected and the fourthpiece of image data is, therefore, not able to be selected from thesegmented image capturing period is described. Here, as illustrated inFIG. 7G, there is assumed to be no image data generated in a segmentedimage capturing period in which image data is not yet selected (a periodindicated by hatched lines in FIG. 7G). In that case, as illustrated inFIG. 7H, the CPU 101 divides each segmented image capturing period intotwo periods. Next, as illustrated in FIG. 7I, the CPU 101 selects thefourth piece of image data from pieces of image data generated insegmented image capturing periods which are other than the segmentedimage capturing periods recognized to contain no image data generatedtherein and in which image data is not yet selected (periods indicatedby solid lines in FIG. 7I). Furthermore, the CPU 101 selects, as thefourth piece of image data, image data (4) with the highest scorespecified in step S410 among the pieces of image data generated in thesegmented image capturing periods.

In step S41106, the CPU 101 acquires a plurality of templatescorresponding to an input entered in the setting screen 301. The CPU 101acquires a plurality of templates having designs corresponding to thecommercial material designation portion 307. Furthermore, an input of,for example, the size of a margin to be left in each spread can bereceived in the setting screen 301, and each template to be acquiredhere can be determined according to, for example, the input margin. Inthis way, in the first exemplary embodiment, pieces of image data to bearranged in each template are automatically selected without the imageprocessing apparatus 100 receiving a selection operation for image datafrom the user.

In step S41107, the CPU 101 determines an image layout of a spreadtargeted for processing, thus generating spread data. Specifically,first, the CPU 101 determines a template suitable for laying out piecesof image data selected in step S41105 from among the plurality oftemplates acquired in step S41106. FIG. 8 is a flowchart illustratingprocessing for determining a template used for generation of a layoutimage in the first exemplary embodiment.

In step S801, the CPU 101 acquires the number of pieces of image dataselected in step S41105 and information about the image data. Theinformation about image data refers to, for example, the width andheight of an image represented by the image data, image capturing dateand time information about the image data, and a score of the imagedata. Furthermore, here, the number of pieces of image data selected instep S41105 is assumed to be three.

In step S802, the CPU 101 selects a template having slots the number ofwhich coincides with the number of pieces of image data selected in stepS41105. Furthermore, since the number of pieces of image data selectedin step S41105 is three, a template having slots the number of which isthree is selected. Here, as a template having slots the number of whichis three, templates (1-1) to (4-4) illustrated in FIG. 9 are assumed tobe selected.

In step S803, the CPU 101 sorts the pieces of image data selected instep S41105 in order of the image capturing date and time. Furthermore,here, the image capturing date and time of each of images respectivelyrepresented by three pieces of image data is assumed to be in such orderas indicated by arrow (A) illustrated in FIG. 9. Moreover, here, animage 905 is assumed to be an image for main slot (a main image), andeach of an image 906 and an image 907 is assumed to be an image forsub-slot (a sub-image).

In step S804, the CPU 101 selects a template suitable for a main imagefrom among the templates selected in step S802. In the present exemplaryembodiment, image data with the earlier image capturing date and time isassumed to be laid out in a slot closer to the upper left corner of atemplate, and image data with the later image capturing date and time isassumed to be laid out in a slot closer to the lower right corner of thetemplate. Therefore, first, the CPU 101 selects a template in which theposition of a main slot corresponds to the order of the image capturingdate and time of image data representing a main image, from among thetemplates selected in step S802. More specifically, for example, in acase where the order of the image capturing date and time of image datarepresenting a main image is the oldest, a template in which the mainslot is the first slot from left of all of the slots is selected.Furthermore, since pieces of image data are sorted in order of the imagecapturing date and time, the CPU 101 is able to recognize the order ofthe image capturing date and time of image data representing a mainimage. Moreover, the CPU 101 finally selects a template in which theaspect ratio of a slot to be used for arranging a main image thereincoincides with (or is close to) the aspect ratio of the main image, fromamong the templates selected in the above-mentioned way. Then, since theimage 905, which is a main image, is an image the image capturing dateand time of which is latest and which is portrait-oriented, here,templates (3-1) to (3-4) illustrated in FIG. 9 are selected.

In step S805, the CPU 101 selects a template suitable for a sub-imagefrom among the templates selected in step S804. Specifically, first, theCPU 101 selects a template in which the position of a sub-slotcorresponds to the order of the image capturing date and time of imagedata representing a sub-image. Moreover, the CPU 101 finally selects atemplate in which the aspect ratio of the sub-slot coincides with (or isclose to) the aspect ratio of the sub-image, from among the templatesselected in the above-mentioned way. Here, the image 906, which is asub-image, is an image the image capturing date and time of which isearliest and which is portrait-oriented. Moreover, the image 907, whichis a sub-image, is an image the image capturing date and time of whichis the second latest and which is landscape-oriented. Therefore, atemplate (3-2) illustrated in FIG. 9 is selected. Furthermore, at thistime, in which sub-slot which sub-image is arranged is also specified.

In this way, the CPU 101 determines a template to be used for generatinga layout image corresponding to a spread and determines in which slot ofthe template which image to lay out, thus generating spread data.Specifically, the CPU 101 manages information about the slots includedin the selected template and pieces of image data corresponding toimages to be laid out in the respective slots, in association with eachother. Thus, spread data includes a template for use in a spread andpieces of image data arranged in the template. In the above-describedway, in the first exemplary embodiment, a template for use in each pageis automatically selected without the image processing apparatus 100receiving a selection operation for image data from the user. Moreover,in the first exemplary embodiment, pieces of image data areautomatically arranged in a template without the image processingapparatus 100 receiving an arrangement operation for image data from theuser. Furthermore, the above-described processing is performed togenerate spread data corresponding to an individual page (individualpage data) and spread data corresponding to a common page (common pagedata). Furthermore, while, in the above description, data is generatedon a spread-by-spread basis, data can be generated on a page-by-pagebasis.

In step S41108, the CPU 101 determines whether the processing in stepsS41105 to S41107 has been completed with respect to all of the pieces ofspread data. Furthermore, in a case where a score for individual pagehas been read in step S41101, the term “all of the pieces of spreaddata” as used in step S41108 refers to all of the pieces of spread datacorresponding to individual pages. Moreover, in a case where a score forcommon page has been read in step S41101, the term “all of the pieces ofspread data” as used in step S41108 refers to all of the pieces ofspread data corresponding to common pages. If it is determined that theprocessing has not yet been completed (NO in step S41108), the CPU 101returns the processing to step S41105, and, then, the CPU 101 performsthe processing in steps S41105 to S41107 with respect to a spread thatis not yet targeted for processing. On the other hand, if it isdetermined that the processing has been completed by the CPU 101 (YES instep S41108), the CPU 101 advances the processing to step S41109.

In step S41109, the CPU 101 determines whether all of the scoresspecified in step S410 (here, scores for individual page concerning thealbum 1, scores for individual page concerning the album 2, and scoresfor common page) have been completely read in step S41101. In otherwords, the CPU 101 determines whether specifying templates with respectto respective individual pages and common pages constituting a pluralityof albums designated by the user to be created has been performed. If itis determined that the processing has not yet been completed (NO in stepS41109), the CPU 101 returns the processing to step S41101, and, then,the CPU 101 reads a score which has not yet been read and performs theprocessing in steps S41102 to S41108. On the other hand, if it isdetermined that the processing has been completed (YES in step S41109),the CPU 101 advances the processing to step S412.

In step S412, the CPU 101 performs spread joining processing for joiningthe generated pieces of spread data and thus generating album data.Here, the CPU 101 creates a layout of the entire album by appending pagenumber information corresponding to a page number in the album to eachpiece of spread data. Moreover, at this time, the CPU 101 arrangespieces of spread data in such a manner that the page number of a page inwhich pieces of image data the image capturing times of which areearlier are arranged becomes smaller, based on the image capturing timesof pieces of image data arranged in a template within each piece ofspread data, and then joins the pieces of spread data. FIG. 10 is aflowchart illustrating the spread joining processing, which is performedby the album creation application according to the first exemplaryembodiment.

In step S1001, the CPU 101 acquires spread data concerning one album outof pieces of spread data generated in step S411. In the first exemplaryembodiment, for example, the term “spread data concerning the album 1”refers to “spread data for common page+spread data for individual pageconcerning the album 1”.

In step S1002, the CPU 101 specifies a value serving as a criterion ofsort (sort criterion value) with respect to each piece of spread dataacquired in step S1001. In the first exemplary embodiment, the sortcriterion value is assumed to be an average image capturing time ofpieces of image data arranged in spread data. Furthermore, the sortcriterion value is not limited to the average image capturing time. Forexample, the sort criterion value can be an average image capturingposition (latitude and longitude) of pieces of image data arranged inspread data, or can be the number of pieces of image data arranged inspread data. Moreover, for example, the sort criterion value can beimage analysis information (for example, color histogram) about piecesof image data arranged in spread data.

In step S1003, the CPU 101 sorts pieces of spread data acquired in stepS1001 based on the sort criterion value specified in step S1002. Since,as mentioned above, in the first exemplary embodiment, the sortcriterion value is an average image capturing time, the CPU 101 arrangespieces of spread data in such a manner that the page number of spreaddata the average image capturing time of which is earlier becomessmaller.

In step S1004, the CPU 101 determines whether the number of pieces ofspread data acquired in step S1001 is equal to the total number ofpieces of spread data. If it is determined that the number of pieces ofspread data acquired in step S1001 is not equal to the total number ofpieces of spread data (NO in step S1004), the CPU 101 advances theprocessing to step S1005. Then, if it is determined that the number ofpieces of spread data acquired in step S1001 is equal to the totalnumber of pieces of spread data (YES in step S1004), the CPU 101advances the processing to step S1006.

In step S1005, the CPU 101 deletes any unnecessary spread data out ofthe pieces of spread data acquired in step S1001, thus performingadjustment in such a manner that the number of pieces of spread dataabout an album targeted for processing becomes the total number ofpieces of spread data. In the first exemplary embodiment, the CPU 101adjusts the number of pieces of spread data by selecting any spread datatargeted for deletion out of pieces of spread data for common page.Specifically, the CPU 101 deletes, out of pieces of spread data forcommon page, spread data the average image capturing time of which isclosest to the average image capturing time of any spread data forindividual page. The CPU 101 repeats this deletion processing, thusadjusting the number of pieces of spread data. Furthermore, the term“average image capturing time” refers to an average of image capturingtimes of a plurality of pieces of image data included in spread data.After that, the CPU 101 returns the processing to step S1004.Furthermore, while, in the first exemplary embodiment, the CPU 101selects spread data targeted for deletion out of pieces of spread datafor common page and does not select spread data targeted for deletionout of pieces of spread data for individual page, the first exemplaryembodiment is not limited to this. For example, the CPU 101 can beconfigured to select spread data targeted for deletion out of pieces ofspread data for individual page and not to select spread data targetedfor deletion out of pieces of spread data for common page. Moreover, forexample, the CPU 101 can be configured to select spread data targetedfor deletion out of both pieces of spread data for individual page andpieces of spread data for common page. When the number of pieces ofspread data has been fixed in the above-mentioned way, the CPU 101 joinsthe pieces of spread data remaining without being deleted in the orderdetermined by sorting performed in step S1003, thus generating albumdata.

In step S1006, the CPU 101 determines whether the processing in stepsS1001 to S1005 has been performed with respect to pieces of spread dataconcerning all of the albums designated to be created via the settingscreen 301. In other words, the CPU 101 determines whether album dataconcerning all of the albums designated to be created via the settingscreen 301 has been generated. If it is determined that the processingin steps S1001 to S1005 has not been performed with respect to pieces ofspread data concerning all of the albums (NO in step S1006), the CPU 101returns the processing to step S1001, in which the CPU 101 acquiresspread data concerning an album which has not yet been acquired and thenperforms processing on the acquired spread data. If it is determinedthat the processing in steps S1001 to S1005 has been performed withrespect to pieces of spread data concerning all of the albums (YES instep S1006), the CPU 101 ends the spread joining processing and advancesthe processing to step S413.

In step S413, the CPU 101 outputs the generated album data. Asoutputting of album data, the CPU 101 is able to perform, for example,displaying of album data on the display 105, transmission of album data,and printing of an album that is based on album data. For example, theCPU 101 is able to transmit generated album data to a printer, thuscausing the printer to print an album that is based on the generatedalbum data.

In the first exemplary embodiment, the user is assumed to be allowed tooptionally edit completed spread data. For example, in a case wherethere is image data which the user desires to change among pieces ofimage data arranged in the respective slots, the user clicks on theimage data which the user desires to change with the mouse 107. Afterthat, the user selects image data to be arranged instead of the imagedata which the user desires to change, so that the image data optionallyselected by the user can be newly arranged instead of the clicked-onimage data. Furthermore, in a case where image data which the userdesires to change has been clicked on, the CPU 101 can display acandidate for image data to be obtained by changing. Moreover, forexample, in a case where image data which the user desires to change isimage data in an individual page, the CPU 101 can display, as acandidate for image data to be obtained by changing, image data thescore for individual page of which is high. Similarly, in a case whereimage data which the user desires to change is image data in a commonpage, the CPU 101 can display, as a candidate for image data to beobtained by changing, image data the score for common page of which ishigh.

In this way, in the first exemplary embodiment, the album creationapplication is able to generate album data for creating an albumincluding a common page and an individual page. Moreover, a criterion ofselecting image data to be arranged in a common page and a criterion ofselecting image data to be arranged in an individual page are controlledto be different from each other. Performing such control enablesselecting image data suitable for a common page as image data to bearranged in a common page and selecting image data suitable for anindividual page as image data to be arranged in an individual page.Moreover, image data to be arranged in a common page and image data tobe arranged in an individual page can be prevented or reduced frombecoming the same.

In the above-described first exemplary embodiment, pieces of image databelonging to the same scene are used for both spread data for commonpage and spread data for individual page. In a second exemplaryembodiment, pieces of image data belonging to the same scene are usedfor only any one of spread data for common page and spread data forindividual page. This prevents or reduces pieces of image data belongingto the same scene from being included in both spread data for commonpage and spread data for individual page.

Furthermore, unless otherwise stated, the configuration of the secondexemplary embodiment is assumed to be similar to the configuration ofthe first exemplary embodiment.

FIGS. 11A, 11B, and 11C are flowcharts illustrating automatic layoutprocessing which is performed by an album creation application accordingto the second exemplary embodiment. The flowcharts illustrated in FIGS.11A to 11C are implemented, for example, by the CPU 101 loading aprogram corresponding to the album creation application stored in theHDD 104 onto the ROM 102 or the RAM 103 and executing the program.

Processing in steps S401 to S410 is similar to the processing describedabove in the first exemplary embodiment.

In step S1101, the CPU 101 specifies for which of a common page and anindividual page each scene is suitable. In other words, the CPU 101specifies which of a score for common page and a score for individualpage is to be applied for each scene. An image data group included in ascene suitable for a common page is a common image data group, and animage data group included in a scene suitable for an individual page isan individual image data group. In the second exemplary embodiment, theCPU 101 specifies which of a score for common page and a score forindividual page is to be applied for each scene according to thefollowing formulae (9) and (10).

$\begin{matrix}{{{}_{}^{}{}_{}^{}} = {\Sigma_{I \in S}{{}_{}^{}{}_{}^{}}}} & (9) \\{{{}_{}^{}{}_{}^{}} = \left\{ \begin{matrix}1 & \left( {{{}_{}^{}{}_{}^{}} > {THR}} \right) \\0 & ({otherwise})\end{matrix} \right.} & (10)\end{matrix}$

^(I)score_(P) denotes the value of a score P assigned to image data I, Sdenotes a scene number allocated to a scene targeted for determination,and THR denotes a threshold value. ^(S)Q_(P) is an index indicating howsuitable for a scene targeted for determination a page corresponding tothe score P is. Furthermore, the score P is one of a score Pd forindividual page and a score Pc for common page.

In a case where the score P is a score for individual page, the CPU 101calculates a value ^(I)T_(Pd), which indicates whether the image data Iis suitable for an individual page, according to formula (10). Then, ina case where the score P is a score for common page, the CPU 101calculates a value ^(I)T_(Pc), which indicates whether the image data Iis suitable for a common page, according to formula (10). As a result,with respect to image data the score for individual page of which ishigher than the threshold value, a value indicating that the image dataI is suitable for an individual page is calculated according to formula(10). Then, with respect to image data the score for common page ofwhich is higher than the threshold value, a value indicating that theimage data I is suitable for a common page is calculated according toformula (10).

After that, in a case where the score P is a score for individual page,the CPU 101 calculates a value ^(S)Q_(Pd), which indicates how manypieces of image data suitable for an individual page are included in ascene targeted for determination, according to formula (9). Moreover, ina case where the score P is a score for common page, the CPU 101calculates a value ^(S)Q_(Pc), which indicates how many pieces of imagedata suitable for a common page are included in a scene targeted fordetermination, according to formula (9).

Then, the CPU 101 specifies for which of a common page and an individualpage a scene targeted for determination is suitable based on the index^(S)Q_(Pc) concerning the score Pc for common page and the index^(S)Q_(Pd) concerning the score Pd for individual page specifiedaccording to formulae (9) and (10). Specifically, the CPU 101 specifiesgenerating spread data in which pieces of image data belonging to thescene targeted for determination are arranged, with use of a scorecorresponding to the larger one in value of the index ^(S)Q_(Pc)concerning the score Pc for common page and the index ^(S)Q_(Pd)concerning the score Pd for individual page. The CPU 101 repeats thepresent processing with respect to all of the scenes and is thus able tospecify which scene is suitable for which of an individual page and acommon page. With this, image data belonging to a scene is used for onlya page suitable for the scene out of a common page and an individualpage. According to the present processing, for example, applying a scorefor common page to scenes (1) to (6) illustrated in FIG. 5B and applyinga score for individual page to scenes (7) to (10) illustrated in FIG. 5Bare specified.

Furthermore, while, in the second exemplary embodiment, a score forindividual page and a score for common page of image data included ineach scene are used as indexes for specifying for which of a common pageand an individual page each scene is suitable, the second exemplaryembodiment is not limited to this. For example, indexes such as thelargeness or smallness of the number of pieces of image data in which amain subject is shown, the largeness or smallness of the number ofpieces of image data in which the number of shown faces is large, andthe number of pieces of image data in each scene can be used. Moreover,for which of a common page and an individual page each scene is suitablecan be specified based on types of scenes determined as a result ofscene segmentation performed in step S406. For example, a sceneclassified into “wedding ceremony” is specified to be suitable for acommon page in which the bride and groom are set as main subjects, and ascene classified into “wedding reception” is specified to be suitablefor an individual page in which family members or friends are set asmain subjects. In this way, information about which type of scene isspecified to be suitable for which page can be previously incorporatedin the album creation application.

Furthermore, while, in the above description, the method of usingformulae (9) and (10) as a method for specifying which scene is suitablefor which of an individual page and a common page has been described,the second exemplary embodiment is not limited to this. For example, thefollowing method can be used to specify which scene is suitable forwhich of an individual page and a common page.

For example, the CPU 101 calculates an average value of scores forcommon page and an average value of scores for individual page of piecesof image data included (contained) in a scene targeted fordetermination. Then, in a case where the average value of scores forcommon page is larger than the average value of scores for individualpage, the CPU 101 determines that the scene targeted for determinationis suitable for a common page. Moreover, in a case where the averagevalue of scores for individual page is larger than the average value ofscores for common page, the CPU 101 determines that the scene targetedfor determination is suitable for an individual page.

Moreover, for example, the CPU 101 calculates the maximum value ofscores for common page of pieces of image data included in a scenetargeted for determination and the maximum value of scores forindividual page of pieces of image data included in the scene targetedfor determination. Then, in a case where the maximum value of scores forcommon page is larger than the maximum value of scores for individualpage, the CPU 101 determines that the scene targeted for determinationis suitable for a common page. Moreover, in a case where the maximumvalue of scores for individual page is larger than the maximum value ofscores for common page, the CPU 101 determines that the scene targetedfor determination is suitable for an individual page.

In step S1102, the CPU 101 generates spread data representing a spreadincluded in an album with use of the scores specified in step S410. FIG.11C is a flowchart illustrating spread data generation processing whichis performed by the album creation application according to the secondexemplary embodiment.

Processing in step S41101 is similar to the processing described abovein the first exemplary embodiment.

In step S110201, the CPU 101 acquires candidate image data, which servesas a candidate for image data to be arranged in a template, from piecesof image data belonging to a scene corresponding to the score read instep S41101, based on the score read in step S41101. In a case where thescore read in step S41101 is a score for individual page, the scenecorresponding to the score read in step S41101 is a scene specified tobe suitable for an individual page in step S1101. Moreover, in a casewhere the score read in step S41101 is a score for common page, thescene corresponding to the score read in step S41101 is a scenespecified to be suitable for a common page in step S1101. Furthermore,in a case where the score read in step S41101 is a score for individualpage, here, candidate image data serving as a candidate for image datato be arranged in a template for individual page is acquired. Moreover,in a case where the score read in step S41101 is a score for commonpage, here, candidate image data serving as a candidate for image datato be arranged in a template for common page is acquired. In the secondexemplary embodiment, the CPU 101 acquires, as candidate image data,image data having a score larger than or equal to a threshold value asthe score read in step S41101. In a case where there is no image datahaving a score larger than or equal to the threshold value, the CPU 101acquires, as candidate image data, image data having the highest score.Furthermore, image data previously selected in image selectionprocessing described below can be excluded from targets for acquisition.

In step S110202, the CPU 101 sets the number of spreads based on aresult of the scene segmentation performed in step S405 and thecandidate image data acquired in step S110201. While the details of thepresent processing are similar to those described in step S41103 in thefirst exemplary embodiment, a difference from the first exemplaryembodiment is described in the following description. In the firstexemplary embodiment, even if the score read in step S41101 is a scorefor individual page or a score for common page, the CPU 101 specifiesthe number of spreads with respect to all of the scenes by applyingformula (5). However, in the second exemplary embodiment, in a casewhere the score read in step S41101 is a score for individual page, theCPU 101 specifies the number of spreads with respect to only a scenespecified to be suitable for an individual page in step S1101. Moreover,in a case where the score read in step S41101 is a score for commonpage, the CPU 101 specifies the number of spreads with respect to only ascene specified to be suitable for a common page in step S1101.

In step S110203, the CPU 101 segments the candidate image data acquiredin step S110201 into image data groups the number of which correspondsto the number of spreads specified in step S110202. Details of thepresent processing are similar to those described in step S41104 in thefirst exemplary embodiment. After that, the CPU 101 advances theprocessing to step S41105.

The subsequent processing is similar to the processing described in thefirst exemplary embodiment.

With such control being performed, in the second exemplary embodiment,pieces of image data belonging to the same scene are controlled to beused for only any one of spread data for common page and spread data forindividual page. This prevents or reduces pieces of image data belongingto the same scene from being included in both spread data for commonpage and spread data for individual page.

As mentioned above, one or a plurality of pieces of image data isincluded in each scene. One image data group included in one scene isreferred to as a “scene image data group”. In the above-described firstexemplary embodiment, both image data used for spread data for commonpage and image data used for spread data for individual page areacquired from each of all of the scene image data groups. In a thirdexemplary embodiment, image data used for spread data for common page isacquired from each of all of the scene image data groups. Then, imagedata used for spread data for individual page is not acquired from eachof all of the scene image data groups but is acquired from only somescene image data groups.

Furthermore, unless otherwise stated, the configuration of the thirdexemplary embodiment is assumed to be similar to the configuration ofthe first exemplary embodiment.

FIGS. 12A, 12B, and 12C are flowcharts illustrating automatic layoutprocessing which is performed by an album creation application accordingto the third exemplary embodiment. The flowcharts illustrated in FIGS.12A to 12C are implemented, for example, by the CPU 101 loading aprogram corresponding to the album creation application stored in theHDD 104 onto the ROM 102 or the RAM 103 and executing the program.

Processing in steps S401 to S410 is similar to the processing describedabove in the first exemplary embodiment.

In step S1201, the CPU 101 specifies whether each scene is suitable foran individual page. In other words, the CPU 101 specifies which of ascore for common page and a score for individual page is to be appliedfor each scene. For example, applying a score for common page to scenes(1) to (8) illustrated in FIG. 5A and applying a score for individualpage to scenes (2) and (5) illustrated in FIG. 5A are specified. In thethird exemplary embodiment, applying a score for common page to all ofthe scenes is specified. Then, the CPU 101 specifies to which scene toapply a score for individual page according to the following formulae(11) and (12), which are the same as formulae (9) and (10) described inthe second exemplary embodiment.

$\begin{matrix}{{{}_{}^{}{}_{}^{}} = {\Sigma_{I \in S}{{}_{}^{}{}_{}^{}}}} & (11) \\{{{}_{}^{}{}_{}^{}} = \left\{ \begin{matrix}1 & \left( {{{}_{}^{}{}_{}^{}} > {THR}} \right) \\0 & ({otherwise})\end{matrix} \right.} & (12)\end{matrix}$

As mentioned in the description of formulae (9) and (10), the CPU 101calculates an index ^(S)Q_(Pd) concerning the score Pd for individualpage. Then, in a case where the index ^(S)Q_(Pd) concerning the score Pdfor individual page is larger than or equal to a threshold value THR2,the CPU 101 determines that the scene targeted for determination issuitable for an individual page. Moreover, in a case where the index^(S)Q_(Pd) concerning the score Pd for individual page is smaller thanthe threshold value THR2, the CPU 101 determines that the scene targetedfor determination is not suitable for an individual page. The CPU 101repeats the present processing with respect to all of the scenes and isthus able to specify which scene is suitable for an individual page.

Furthermore, while, in the third exemplary embodiment, a score forindividual page of image data included in each scene is used as an indexfor specifying whether each scene is suitable for an individual page,the third exemplary embodiment is not limited to this. For example,indexes such as the largeness or smallness of the number of pieces ofimage data in which a main subject is shown, the largeness or smallnessof the number of pieces of image data in which the number of shown facesis large, and the number of pieces of image data in each scene can beused. Moreover, whether each scene is suitable for an individual pagecan be specified based on types of scenes determined as a result ofscene segmentation performed in step S406. For example, a sceneclassified into “wedding reception” is specified to be suitable for anindividual page in which family members or friends are set as mainsubjects. In this way, information about which type of scene isspecified to be suitable for an individual page can be previouslyincorporated in the album creation application.

In step S1202, the CPU 101 generates spread data representing a spreadincluded in an album with use of the scores specified in step S410. FIG.12C is a flowchart illustrating spread data generation processing whichis performed by the album creation application according to the thirdexemplary embodiment.

Processing in step S41101 is similar to the processing described abovein the first exemplary embodiment.

In step S120201, the CPU 101 acquires candidate image data, which servesas a candidate for image data to be arranged in a template, from piecesof image data belonging to a scene corresponding to the score read instep S41101, based on the score read in step S41101. In a case where thescore read in step S41101 is a score for individual page, the scenecorresponding to the score read in step S41101 is a scene specified tobe suitable for an individual page in step S1201. Moreover, in a casewhere the score read in step S41101 is a score for common page, thescene corresponding to the score read in step S41101 is each of all ofthe scenes. Furthermore, in a case where the score read in step S41101is a score for individual page, here, candidate image data serving as acandidate for image data to be arranged in a template for individualpage is acquired. Moreover, in a case where the score read in stepS41101 is a score for common page, here, candidate image data serving asa candidate for image data to be arranged in a template for common pageis acquired. In the third exemplary embodiment, the CPU 101 acquires, ascandidate image data, image data having a score larger than or equal toa threshold value as the score read in step S41101. In a case wherethere is no image data having a score larger than or equal to thethreshold value, the CPU 101 acquires, as candidate image data, imagedata having the highest score. Furthermore, image data previouslyselected in image selection processing described below can be excludedfrom targets for acquisition.

Moreover, in the third exemplary embodiment, the CPU 101 selectscandidate image data according to the following formula (13).

$\begin{matrix}{{{}_{}^{}{}_{}^{}} = \left\{ \begin{matrix}1 & {\left( {{{{{{I \in {SI}_{P}}\mspace{14mu}\&}\mspace{20mu}{{{}_{}^{}{}_{p = {P\; 1}}^{}} \geq}{THR}}\mspace{14mu}\&}\mspace{14mu}{{}_{}^{}{}_{p \neq {P\; 2}}^{}}} < {{}_{}^{}{}_{p = {P\; 1}}^{}}}\; \right)\;} \\0 & ({otherwise})\end{matrix} \right.} & (13)\end{matrix}$

^(I)C_(P) denotes a value indicating whether image data I is selected ascandidate image data in a case where the score read in step S41101 isany one score P of a score P1 for individual page and a score P2 forcommon page. SI denotes an aggregation of pieces of image data belongingto a scene S corresponding to the score P. ^(I)score_(p=P1) and^(I)Score_(p≠P1) denote a score P1 for individual page of the image dataI and a score other than the score P1 for individual page of the imagedata I (i.e., a score P2 for common page), respectively. THR denotes athreshold value. In a case where the value ^(I)C_(P) is 1, the value^(I)C_(P) indicates that the image data I is selected as candidate imagedata, and, in a case where the value ^(I)C_(P) is 0, the value ^(I)C_(p)indicates that the image data I is not selected as candidate image data.

With this, image data the score for individual page of which is higherthan the score for common page thereof is selected as candidate imagedata serving as a candidate for image data to be arranged in a templatefor individual page. Moreover, image data the score for common page ofwhich is higher than the score for individual page thereof is selectedas candidate image data serving as a candidate for image data to bearranged in a template for common page. Employing such a configurationenables preventing or reducing the same image data from being used forboth an individual page and a common page. Furthermore, the method forselecting image data according to the above-mentioned formulae can alsobe applied in, for example, the first exemplary embodiment and thesecond exemplary embodiment.

In step S120202, the CPU 101 sets the number of spreads based on aresult of the scene segmentation performed in step S405 and thecandidate image data acquired in step S120201. While the details of thepresent processing are similar to those described in step S41103 in thefirst exemplary embodiment, a difference from the first exemplaryembodiment is described in the following description. In the firstexemplary embodiment, even if the score read in step S41101 is a scorefor individual page or a score for common page, the CPU 101 specifiesthe number of spreads with respect to all of the scenes by applyingformula (5). However, in the third exemplary embodiment, in a case wherethe score read in step S41101 is a score for individual page, the CPU101 specifies the number of spreads with respect to only a scenespecified to be suitable for an individual page in step S1201. On theother hand, in a case where the score read in step S41101 is a score forcommon page, the CPU 101 specifies the number of spreads with respect toall of the scenes.

In step S120203, the CPU 101 segments the candidate image data acquiredin step S120201 into image data groups the number of which correspondsto the number of spreads specified in step S120202. Details of thepresent processing are similar to those described in step S41104 in thefirst exemplary embodiment. After that, the CPU 101 advances theprocessing to step S41105.

The subsequent processing is similar to the processing described in thefirst exemplary embodiment.

In this way, in the third exemplary embodiment, since all of the scenesserve as a candidate for selecting an image for common page, as comparedwith a configuration in which only some scenes serve as a candidate forselecting an image for common page, the possibility of a better imagebeing selected as an image for common page increases. Moreover, sinceonly some scenes serve as a candidate for selecting an image forindividual page, an image selected as an image for common page and animage selected as an image for individual page can be prevented orreduced from becoming the same image.

Furthermore, in the third exemplary embodiment, image data used forspread data for common page is acquired from each of all of the sceneimage data groups, and image data used for spread data for individualpage is not acquired from each of all of the scene image data groups butis acquired from only some scene image data groups. However, the thirdexemplary embodiment is not limited to this configuration. For example,there is a case where the number of pieces of image data suitable forspread data for individual page is small. In such a case, for example,image data used for spread data for individual page can be acquired fromeach of all of the scene image data groups. Moreover, image data usedfor spread data for common page can be configured not to be acquiredfrom all of the scene image data groups but to be acquired from onlysome scene image data groups. With this, while a sufficient number ofpieces of image data used for spread data for individual page aresecured, an image selected as an image for common page and an imageselected as an image for individual page can be prevented or reducedfrom becoming the same.

In the above-described exemplary embodiments, album data is generatedwithout a page corresponding to a cover and the other pages beingdistinguished from each other. In a fourth exemplary embodiment, albumdata is generated with a page corresponding to a cover and the otherpages being distinguished from each other. Additionally, in the fourthexemplary embodiment, a configuration in which a page corresponding to acover is controlled to be a common page and a configuration in which apage corresponding to a cover is controlled to be an individual page aredescribed.

Furthermore, unless otherwise stated, the configuration of the fourthexemplary embodiment is assumed to be similar to the configuration ofthe first exemplary embodiment.

In the fourth exemplary embodiment, the album creation applicationfurther displays, on the setting screen 301, a cover mode setting area1400 for receiving setting of a mode for cover, such as that illustratedin FIG. 14A. An area 1401 is an area for selecting an individual mode asthe mode for cover, and an area 1402 is an area for selecting a commonmode as the mode for cover. The individual mode is a mode to performcontrol in such a manner that a page corresponding to a cover is anindividual page, and the common mode is a mode to perform control insuch a manner that a page corresponding to a cover is a common page.Furthermore, while the mode for cover selected here is applied to all ofthe albums designated to be created via the setting screen 301, thefourth exemplary embodiment is not limited to this, and the mode forcover can be selected for each album.

FIGS. 13A, 13B, 13C, and 13D are flowcharts illustrating automaticlayout processing which is performed by the album creation applicationaccording to the fourth exemplary embodiment. The flowcharts illustratedin FIGS. 13A to 13D are implemented, for example, by the CPU 101 loadinga program corresponding to the album creation application stored in theHDD 104 onto the ROM 102 or the RAM 103 and executing the program.

Processing in steps S401 to S411 is similar to the processing describedabove in the first exemplary embodiment.

In step S1301, the CPU 101 performs cover data generation processing fordetermining a layout of the cover and generating cover data used toprint the cover with the determined layout. Furthermore, in the fourthexemplary embodiment, the cover data is assumed to be data used to printpages respectively corresponding to a front cover and a back cover.However, the fourth exemplary embodiment is not limited to thisconfiguration, and, for example, the cover data can be data used toprint a page corresponding to only a front cover. FIG. 13D is aflowchart illustrating cover data generation processing which isperformed by the album creation application according to the fourthexemplary embodiment.

In step S130101, the CPU 101 specifies cover data of which album togenerate out of a plurality of albums designated to be created via thesetting screen 301.

In step S130102, the CPU 101 specifies a mode for cover of the albumspecified in step S130101 based on an input entered by the user in thecover mode setting area 1400.

In step S130103, the CPU 101 acquires cover candidate image data servingas a candidate for image data to be arranged in a cover of the albumspecified in step S130101. Specifically, in a case where the mode forcover of the album specified in step S130101 is the individual mode, theCPU 101 acquires, as cover candidate image data, image data included inspread data corresponding to an individual page out of pieces of spreaddata generated in step S411. Then, in a case where the mode for cover ofthe album specified in step S130101 is the common mode, the CPU 101acquires, as cover candidate image data, image data included in spreaddata corresponding to a common page out of pieces of spread datagenerated in step S411.

In step S130104, the CPU 101 selects image data to be arranged in thecover from the cover candidate image data. In the fourth exemplaryembodiment, since the number of pieces of image data to be arranged inthe cover is one, the CPU 101 selects only one piece of image data.Specifically, in a case where the mode for cover of the album specifiedin step S130101 is the individual mode, the CPU 101 selects image datathe score for individual page of which is high from the cover candidateimage data. Furthermore, the score for individual page to be referred toat this time is a score for individual page evaluated based on a mainsubject set in the album specified in step S130101. Moreover, in a casewhere the mode for cover of the album specified in step S130101 is thecommon mode, the CPU 101 selects image data the score for common page ofwhich is high from the cover candidate image data. Furthermore, if aplurality of pieces of image data is to be arranged in the cover, aplurality of pieces of image data can be selected in step S130104.

Furthermore, while, in the above description, image data to be arrangedin the cover is selected from pieces of image data included in spreaddata generated in step S411, the fourth exemplary embodiment is notlimited to this configuration. As long as image data to be selected isimage data corresponding to the mode for cover (i.e., the scorecorresponding to the mode for cover is high), image data that is notincluded in spread data generated in step S411 can be selected.

Moreover, image data can be selected based on a factor other than ascore corresponding to the mode for cover. For example, image data inwhich only a subject image is shown can be selected in preference toimage data in which a person other than a main subject is shown.Moreover, image data conforming to the design or aspect ratio of a slotarranged in a template for cover can be selected in preference to imagedata not conforming thereto. Moreover, image data the resolution ofwhich becomes greater than or equal to a predetermined value when theimage data is arranged in a slot can be preferentially selected.

In step S130105, the CPU 101 acquires a template for cover. In thefourth exemplary embodiment, only one template for cover is assumed tobe previously prepared. An example of the template for cover acquired instep S130105 is illustrated in FIG. 15A. In the templates for coverillustrated in FIGS. 15A, 15B, 15C, and 15D, a region on the right sideof the central line is a region corresponding to a front cover, and aregion on the left side of the central line is a region corresponding toa back cover. For example, a slot 1502 is arranged in a regioncorresponding to the front cover of the template for cover 1501, and anyslot is not arranged in a region corresponding to the back cover of thetemplate for cover 1501. Furthermore, in step S130105, a plurality oftemplates for cover different in type from each other can be acquired.Moreover, a slot can also be arranged in a region corresponding to theback cover. Furthermore, the templates for cover such as thoseillustrated in FIGS. 15A to 15D and the templates for page other thantemplates for cover such as those illustrated in FIG. 9 are assumed tobe different from each other in the layout such as the size of each slotor the number of slots. Moreover, a region in which to arrange text suchas an album name can be provided in a template for cover.

In step S130106, the CPU 101 generates cover data. Specifically, the CPU101 generates cover data by arranging the image data selected in stepS130104 in a slot of the template acquired in step S130105. Furthermore,in a case where a plurality of templates for cover has been acquired instep S130105, the CPU 101 can determine which of the acquired pluralityof templates for cover to use or in which slot to arrange which ofpieces of image data, in a similar way to the processing illustrated inFIG. 8. Furthermore, in a case where a slot is also arranged in a regioncorresponding to the back cover, the CPU 101 also arranges the imagedata selected in step S130104 in the slot arranged in the regioncorresponding to the back cover. Furthermore, for example, even in acase where the mode for cover is the common mode, image data forindividual page (image data the score for individual page of which ishigh) can be arranged in the slot arranged in the region correspondingto the back cover.

In step S130107, the CPU 101 determines whether generation of cover datahas been completed with respect to all of the albums designated to becreated via the setting screen 301. If it is determined that thegeneration of cover data has not yet been completed (NO in stepS130107), the CPU 101 returns the processing to step S130101, in whichthe CPU 101 specifies generating cover data of an album not yetsubjected to generation of cover data. On the other hand, if it isdetermined that the generation of cover data has been completed (YES instep S130107), the CPU 101 advances the processing to step S1302.

In step S1302, the CPU 101 performs spread joining processing forjoining the generated cover data and pieces of spread data and thusgenerating album data. While the present processing is similar to thatin step S412 described in the first exemplary embodiment, the CPU 101arranges the cover data and pieces of spread data in such a manner thatthe page number of a page corresponding to the front cover is thesmallest and the page number of a page corresponding to the back coveris the largest and then joins the arranged cover data and pieces ofspread data. At this time, although pieces of spread data other than thecover data are arranged based on the sort criterion value, the coverdata is arranged irrespective of the sort criterion value. Moreover, inadjustment of the number of pieces of spread data, cover data iscontrolled in such a way as not to be targeted for deletion. After that,the CPU 101 advances the processing to step S413.

The subsequent processing is similar to the processing described in thefirst exemplary embodiment.

With the above-described configuration employed, album data can begenerated with a page corresponding to a cover and the other pagesdistinguished from each other. Moreover, an image suitable for a covercan be arranged in a template for cover as an image to be included inthe cover.

Furthermore, while, in the above description, a configuration in whichthe user is allowed to select the individual mode and the common modehas been described, the fourth exemplary embodiment is not limited tothis configuration. For example, a configuration in which only apredetermined one of the individual mode and the common mode is used togenerate cover data can also be employed. In that case, displaying acover mode setting area to receive selection of the mode for cover fromthe user does not need to be performed.

In a fifth exemplary embodiment, a configuration for receiving settingof the mode for cover in more detail is described.

Furthermore, unless otherwise stated, the configuration of the fifthexemplary embodiment is assumed to be similar to the configuration ofthe first exemplary embodiment.

In the fifth exemplary embodiment, the album creation applicationfurther displays, on the setting screen 301, a cover mode setting area1410 for receiving setting of the mode for cover, such as thatillustrated in FIG. 14B. In the fifth exemplary embodiment, three areas(areas 1411 to 1413) are assumed to be included in areas used to selectthe individual mode as the mode for cover. The area 1411 is similar tothe area 1401. The area 1412 is an area used to select an individualmode for placing, on the back cover, an image including a main subjectset in another album. The area 1413 is an area used to select anindividual mode for placing, on the front cover, an image including amain subject set in another album. An area 1414 is similar to the area1402. Furthermore, while the mode for cover selected here is applied toall of the albums designated to be created via the setting screen 301,the fifth exemplary embodiment is not limited to this, and the mode forcover can be selected for each album.

The album data generation processing performed in a case where the area1411 has been selected is similar to the album data generationprocessing performed in a case where the area 1401 has been selected.Moreover, the album data generation processing performed in a case wherethe area 1414 has been selected is similar to the album data generationprocessing performed in a case where the area 1402 has been selected.

Processing which is performed in a case where the area 1412 has beenselected is described. In step S130104, the CPU 101 selects image datathe score for individual page of which evaluated based on a main subjectset in an album specified in step S130101 is high, as in the fourthexemplary embodiment. Then, additionally, the CPU 101 selects image datathe score for individual page of which evaluated based on a main subjectset in an album other than the album specified in step S130101 is high.

In step S130105, the CPU 101 acquires a template for cover 1503 (FIG.15B) corresponding to the area 1412.

In step S130106, the CPU 101 places, in a slot 1504 in the front cover,the image data the score for individual page of which evaluated based ona main subject set in an album specified in step S130101 is high. Then,the CPU 101 places, in a slot 1505 and a slot 1506 in the back cover,the image data the score for individual page of which evaluated based ona main subject set in an album other than the album specified in stepS130101 is high. This enables, while making a main subject set in analbum specified in step S130101 conspicuous, increasing variations oflayout in the back cover of the album.

The subsequent processing is similar to the processing described in thefourth exemplary embodiment.

Processing which is performed in a case where the area 1413 has beenselected is described. In step S130104, the CPU 101 selects image datathe score for individual page of which evaluated based on a main subjectset in an album specified in step S130101 is high, as in the fourthexemplary embodiment. Then, additionally, the CPU 101 selects image datathe score for individual page of which evaluated based on a main subjectset in an album other than the album specified in step S130101 is high.

In step S130105, the CPU 101 acquires a template for cover 1507 (FIG.15C) corresponding to the area 1413.

In step S130106, the CPU 101 places, in a slot 1508 in the front cover,the image data the score for individual page of which evaluated based ona main subject set in an album specified in step S130101 is high. Then,the CPU 101 places, in a slot 1509 and a slot 1510 in the front cover,the image data the score for individual page of which evaluated based ona main subject set in an album other than the album specified in stepS130101 is high. The slot 1508 is large as compared with the slot 1509or the slot 1510. This enables, while making a main subject set in analbum specified in step S130101 conspicuous, increasing variations oflayout in the front cover of the album.

The subsequent processing is similar to the processing described in thefirst exemplary embodiment.

In a sixth exemplary embodiment, a configuration for performingprocessing different from the processing performed in theabove-described exemplary embodiments, as processing performed in a casewhere the common mode has been selected as a mode for cover, isdescribed.

In the above-described exemplary embodiments, in a case where the modefor cover is the common mode, the CPU 101 selects image data the scorefor common page of which is high from the cover candidate image data.However, in the sixth exemplary embodiment, in a case where the mode forcover is the common mode, the CPU 101 selects a plurality of pieces ofimage data the score for individual page is high from the covercandidate image data. Specifically, for example, suppose that creationof three albums, i.e., an album in which a main subject A is set, analbum in which a main subject B is set, and an album in which a mainsubject C is set, has been designated. In that case, in step S130104,the CPU 101 selects image data the score for individual page of whichevaluated based on the main subject A is high, image data the score forindividual page of which evaluated based on the main subject B is high,and image data the score for individual page of which evaluated based onthe main subject C is high. Furthermore, the number of pieces of imagedata to be selected is not limited to three. Pieces of image data thenumber of which is larger than or equal to at least the number of albumsdesignated to be created only needs to be selected.

In step S130105, the CPU 101 acquires a template for cover 1511 (FIG.15D) for the common mode.

In step S130106, the CPU 101 places, in a slot 1512 to a slot 1514, therespective pieces of image data selected in step S130104. The sizes ofthe slot 1512 to the slot 1514 are assumed to be the same. Moreover, inwhich slot which piece of image data is placed is not particularlylimited. For example, image data the score for individual page of whichevaluated based on a main subject in an album specified in step S130101is high can be placed in the slot 1513, which is located at the centerof the front cover area.

In this way, in the sixth exemplary embodiment, a cover for the commonmode is configured by placing, in the cover, a plurality of pieces ofimage data corresponding to the respective main subjects. Furthermore,in a case where the individual mode has been described as the mode forcover, as described in the fourth exemplary embodiment and the fifthexemplary embodiment, at least one piece of image data corresponding toone main subject is placed in the cover.

In the above-described exemplary embodiments, a configuration in whichthe number of pieces of spread data corresponding to an individual pagediffers depending on albums has been described. However, the exemplaryembodiments are not limited to this configuration, and the number ofpieces of spread data corresponding to an individual page can becontrolled to be the same with respect to all of a plurality of albumsdesignated by the user to be created.

In the above-described exemplary embodiments, a configuration in whichthe number of pieces of spread data corresponding to a common pagediffers depending on albums has been described. However, the exemplaryembodiments are not limited to this configuration, and the number ofpieces of spread data corresponding to a common page can be controlledto be the same with respect to all of a plurality of albums designatedby the user to be created.

In the above-described exemplary embodiments, an aggregation of piecesof spread data for common page and pieces of spread data for individualpage is sorted according to the sort criterion value, so that album datais generated. Thus, in which position (page number) spread data forcommon page or spread data for individual page is placed in album datadiffers for each piece of album data. However, the exemplary embodimentsare not limited to this configuration, and, for example, the position ofspread data for common page and the position of spread data forindividual page can be fixed in each piece of album data. For example,pieces of spread data can be arranged in such a manner that spread datafor individual page is placed after spread data for common page. In moredetail, pieces of spread data can be arranged in such a manner thatpieces of spread data for individual page sorted according to the sortcriterion value are placed after pieces of spread data for common pagesorted according to the sort criterion value. Naturally, pieces ofspread data can also be arranged in such a manner that spread data forcommon page is placed after spread data for individual page.

The above-described exemplary embodiments can also be implemented byperforming the following processing. More specifically, software(program) for implementing the functions of the above-describedexemplary embodiments is supplied to a system or apparatus via a networkor any type of storage medium, and a computer (or a CPU or microprocessing unit (MPU)) of the system or apparatus reads out and executesthe program to perform the processing. Moreover, the program can beexecuted by a single computer or can be executed by a plurality ofcomputers operating in conjunction with each other. Moreover, not all ofthe above-described processing operations need to be implemented bysoftware, but a part or all of the above-described processing operationscan be implemented by hardware such as an application specificintegrated circuit (ASIC). Moreover, the CPU includes not only a singleCPU performing all of the processing operations but also a plurality ofCPUs performing the processing operations in cooperation with each otheras appropriate. Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random access memory (RAM), a read-only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

What is claimed is:
 1. A control method for an image processingapparatus that generates a plurality of pieces of album datacorresponding to a plurality of albums, the control method comprising:acquiring a plurality of pieces of image data; evaluating each of theplurality of pieces of image data based on a first factor; evaluatingeach of the plurality of pieces of image data based on a second factordifferent from the first factor; automatically selecting at least onepiece of image data out of the plurality of pieces of image data ascommon image data corresponding to an image to be placed in a commonpage, which is a page common to all of the plurality of albums, based onan evaluation that is based on the first factor; generating common pagedata corresponding to the common page based on the at least one piece ofimage data selected as the common image data; automatically selecting atleast one piece of image data out of the plurality of pieces of imagedata as individual image data corresponding to an image to be placed inan individual page, which is a page not common to all of the pluralityof albums, based on an evaluation that is based on the second factor;generating a plurality of pieces of individual page data correspondingto the individual page based on the at least one piece of image dataselected as the individual image data; and generating the plurality ofpieces of album data based on the common page data and the plurality ofpieces of individual page data, wherein the individual page datacorresponding to first album data out of the plurality of pieces ofalbum data and the individual page data corresponding to second albumdata out of the plurality of pieces of album data are different fromeach other, and wherein the common page data corresponding to the firstalbum data out of the plurality of pieces of album data and the commonpage data corresponding to the second album data out of the plurality ofpieces of album data are identical to each other.
 2. The control methodaccording to claim 1, wherein each of the plurality of pieces of imagedata is evaluated by the second factor concerning the first album dataand the second factor concerning the second album data, wherein at leastone piece of image data out of the plurality of pieces of image data isautomatically selected as the individual image data corresponding to thefirst album data based on an evaluation that is based on the secondfactor concerning the first album data, wherein at least one piece ofimage data out of the plurality of pieces of image data is automaticallyselected as the individual image data corresponding to the second albumdata based on an evaluation that is based on the second factorconcerning the second album data, wherein the individual page datacorresponding to the first album data is generated based on the at leastone piece of image data selected as the individual image datacorresponding to the first album data, wherein the individual page datacorresponding to the second album data is generated based on the atleast one piece of image data selected as the individual image datacorresponding to the second album data, wherein the album datacorresponding to the first album data is generated based on the commonpage data and the individual page data corresponding to the first albumdata, and wherein the album data corresponding to the second album datais generated based on the common page data and the individual page datacorresponding to the second album data.
 3. The control method accordingto claim 2, further comprising specifying at least one main subjectrespectively corresponding to the plurality of pieces of album data,wherein the first factor includes a factor concerning the at least onemain subject corresponding to the first album data and a factorconcerning the at least one main subject corresponding to the secondalbum data, wherein the second factor concerning the first album dataincludes a factor concerning the at least one main subject correspondingto the first album data and does not include a factor concerning the atleast one main subject corresponding to the second album data, andwherein the second factor concerning the second album data includes afactor concerning the at least one main subject corresponding to thesecond album data and does not include a factor concerning the at leastone main subject corresponding to the first album data.
 4. The controlmethod according to claim 3, wherein the first factor includes a factorconcerning a size of the at least one main subject corresponding to thefirst album data and a factor concerning a size of the at least one mainsubject corresponding to the second album data, wherein the secondfactor concerning the first album data includes a factor concerning asize of the at least one main subject corresponding to the first albumdata and does not include a factor concerning a size of the at least onemain subject corresponding to the second album data, and wherein thesecond factor concerning the second album data includes a factorconcerning a size of the at least one main subject corresponding to thesecond album data and does not include a factor concerning a size of theat least one main subject corresponding to the first album data.
 5. Thecontrol method according to claim 3, wherein the first factor includes afactor concerning whether the at least one main subject corresponding tothe first album data is included and a factor concerning whether the atleast one main subject corresponding to the second album data isincluded, wherein the second factor concerning the first album dataincludes a factor concerning whether the at least one main subjectcorresponding to the first album data is included and does not include afactor concerning whether the at least one main subject corresponding tothe second album data is included, and wherein the second factorconcerning the second album data includes a factor concerning whetherthe at least one main subject corresponding to the second album data isincluded and does not include a factor concerning whether the at leastone main subject corresponding to the first album data is included. 6.The control method according to claim 3, wherein the first factorincludes a factor concerning whether the at least one main subjectcorresponding to the first album data is a smiling subject and a factorconcerning whether the at least one main subject corresponding to thesecond album data is a smiling subject, wherein the second factorconcerning the first album data includes a factor concerning whether theat least one main subject corresponding to the first album data is asmiling subject and does not include a factor concerning whether the atleast one main subject corresponding to the second album data is asmiling subject, and wherein the second factor concerning the secondalbum data includes a factor concerning whether the at least one mainsubject corresponding to the second album data is a smiling subject anddoes not include a factor concerning whether the at least one mainsubject corresponding to the first album data is a smiling subject. 7.The control method according to claim 2, wherein each of the firstfactor and the second factor includes a factor concerning image quality.8. The control method according to claim 1, further comprising:segmenting the plurality of pieces of image data into a plurality ofimage data groups according to a predetermined criterion; specifying atleast one image data group out of the plurality of image data groups asa common image data group for selecting the common image data; andspecifying at least one image data group out of the plurality of imagedata groups as an individual image data group for selecting theindividual image data, wherein at least one piece of image data out ofthe at least one image data group specified as the common image datagroup is selected as the common image data, and wherein at least onepiece of image data out of the at least one image data group specifiedas the individual image data group is selected as the individual imagedata.
 9. The control method according to claim 8, wherein at least oneimage data group out of the at least one image data group specified asthe common image data group is controlled in such a way as not to bespecified as the individual image data group.
 10. The control methodaccording to claim 9, wherein all of the image data groups out of the atleast one image data group specified as the common image data group iscontrolled in such a way as not to be specified as the individual imagedata group.
 11. The control method according to claim 9, wherein all ofthe image data groups out of the plurality of image data groups arespecified as the common image data group, and wherein at least one imagedata group out of the plurality of image data groups is not specified asthe individual image data group.
 12. The control method according toclaim 1, wherein a first evaluation value is appended to the pluralityof pieces of image data based on an evaluation that is based on thefirst factor, wherein a second evaluation value is appended to theplurality of pieces of image data based on an evaluation that is basedon the second factor, wherein at least one piece of image data out ofthe plurality of pieces of image data is automatically selected as thecommon image data based on the first evaluation value, and wherein atleast one piece of image data out of the plurality of pieces of imagedata is automatically selected as the individual image data based on thesecond evaluation value.
 13. The control method according to claim 12,wherein at least one piece of image data to which the first evaluationvalue larger than that of a different piece of image data has beenappended out of the plurality of pieces of image data is automaticallyselected as the common image data, and wherein at least one piece ofimage data to which the second evaluation value larger than that of adifferent piece of image data has been appended out of the plurality ofpieces of image data is automatically selected as the individual imagedata.
 14. The control method according to claim 13, wherein at least onepiece of image data to which the first evaluation value larger than thatof a different piece of image data has been appended and to which thefirst evaluation value larger in value than the second evaluation valuehas been appended out of the plurality of pieces of image data isautomatically selected as the common image data, and wherein at leastone piece of image data to which the second evaluation value larger thanthat of a different piece of image data has been appended and to whichthe second evaluation value larger in value than the first evaluationvalue has been appended out of the plurality of pieces of image data isautomatically selected as the individual image data.
 15. The controlmethod according to claim 1, further comprising performing control insuch a manner that cover data corresponding to a cover serves as thecommon page data.
 16. The control method according to claim 1, furthercomprising performing control in such a manner that cover datacorresponding to a cover serves as the individual page data.
 17. Thecontrol method according to claim 1, further comprising receiving aninput entered by a user concerning cover data corresponding to a cover,wherein cover data corresponding to a cover is controlled in such a wayas to serve as data that is based on the input out of the common pagedata and the individual page data.
 18. The control method according toclaim 1, wherein the common page data is generated by the at least onepiece of image data selected as the common image data beingautomatically placed in a first template without reception of a placingoperation performed by a user, and wherein the individual page data isgenerated by the at least one piece of image data selected as theindividual image data being automatically placed in a second templatewithout reception of a placing operation performed by a user.
 19. Thecontrol method according to claim 1, further comprising outputting theplurality of pieces of album data, wherein outputting of the pluralityof pieces of album data is at least one of displaying of the pluralityof pieces of album data, transmission of the plurality of pieces ofalbum data, and printing of the plurality of albums that is based on theplurality of pieces of album data.
 20. A non-transitorycomputer-readable storage medium storing computer-executableinstructions that, when executed by a computer of an image processingapparatus that generates a plurality of pieces of album datacorresponding to a plurality of albums, cause the computer to perform amethod comprising: acquiring a plurality of pieces of image data;evaluating each of the plurality of pieces of image data based on afirst factor; evaluating each of the plurality of pieces of image databased on a second factor different from the first factor; automaticallyselecting at least one piece of image data out of the plurality ofpieces of image data as common image data corresponding to an image tobe placed in a common page, which is a page common to all of theplurality of albums, based on an evaluation that is based on the firstfactor; generating common page data corresponding to the common pagebased on the at least one piece of image data selected as the commonimage data; automatically selecting at least one piece of image data outof the plurality of pieces of image data as individual image datacorresponding to an image to be placed in an individual page, which is apage not common to all of the plurality of albums, based on anevaluation that is based on the second factor; generating a plurality ofpieces of individual page data corresponding to the individual pagebased on the at least one piece of image data selected as the individualimage data; and generating the plurality of pieces of album data basedon the common page data and the plurality of pieces of individual pagedata, wherein the individual page data corresponding to first album dataout of the plurality of pieces of album data and the individual pagedata corresponding to second album data out of the plurality of piecesof album data are different from each other, and wherein the common pagedata corresponding to the first album data out of the plurality ofpieces of album data and the common page data corresponding to thesecond album data out of the plurality of pieces of album data areidentical to each other.